Overt vs covert penetration testing

Overt vs covert penetration testing: which one to buy first

For a first test, buy overt penetration testing. If your organization has never had a penetration test, or has only run scans, an overt engagement with IT and security staff informed gives you a list of exploitable weaknesses you can fix and retest. NIST SP 800-115 describes overt testing as less expensive, carrying less risk and more frequently used than covert work, and it lets staff watch how assessors evaluate and circumvent controls, which is useful training for the team that will fix the findings. Covert work only makes sense once the basic weaknesses are known and closed, because a stealth exercise against an unhardened environment mostly reproduces findings you already have. Ask any provider to confirm in writing whether the first engagement is overt, and what access and boundaries it will use.

Buy covert work when the question is detection and response. NIST SP 800-115 frames covert testing as examining the damage an adversary could cause, not finding every vulnerability. It is conducted without the IT staff's knowledge, with upper management's permission, and it tests technical controls, how staff respond to perceived incidents and whether policies are known in practice. Because stealth requirements make it time-consuming and costly, pay for it when you already have a mature program and a specific concern, such as whether your monitoring would notice an intruder who moves slowly. Before scoping, confirm with the provider who holds the stop conditions, what access level ends the test, how a detection is handled, and how the results will be reported to the people who did not know the test was running.

Combine the two once both have a purpose. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk. A practical sequence is an overt test to close known weaknesses, followed by covert or red team work that checks whether your staff detect and contain what remains. Plan for the combined program in writing, including who in management approves each covert window, so that a test is never mistaken for a real incident. Cost follows scope: covert work adds stealth effort on top of the testing itself, so ask quotes to show the two engagements separately.

How overt and covert penetration testing compare on each factor

This table compares overt and covert penetration testing on goal, staff awareness, permissions, cost, risk and typical outputs, drawn from NIST SP 800-115. Read each row as a trade-off: overt work is the usual starting point, while covert work tests whether detection and response hold up under realistic conditions.

Factor	Overt penetration testing	Covert penetration testing
Primary goal	Find and validate as many vulnerabilities as the agreed scope allows	Examine the damage an adversary could cause, not find every vulnerability
What is measured	Technical weaknesses in the defined targets	Technical controls, staff response to perceived incidents and policy knowledge
IT and security staff awareness	Aware and involved, so they can observe the assessors' methods	Unaware of the test, so detection and response are part of the result
Management permission	Agreed with management before work starts	Requires upper management's permission while staff are kept unaware
Stealth requirements	Low, because the testers work openly with the team	High, which NIST SP 800-115 describes as making covert testing time-consuming and costly
Risk to systems	Lower, as NIST SP 800-115 describes overt testing as carrying less risk	Higher or comparable to overt work when exploitation touches production, so limits must be agreed
Relative cost	Less expensive, per NIST SP 800-115	More expensive, because stealth and extended evasion add effort
Boundaries	Set in the rules of engagement, typically by target and technique	Stops at defined boundaries such as a certain level of access
Typical outputs	Vulnerability findings with validated exploitation and remediation guidance	Impact narrative, detection gaps and response timeline review
Suitability for a first test	Usually the better first test, to establish a baseline before adding stealth	Usually unsuited to a first test, since staff need a baseline and tested controls first

How overt and covert testing run day to day

In an overt test, the IT and security staff know the test is happening and are involved in it. Management approves the scope, and the testers work through the agreed targets in the open, often with a named contact who can answer questions or pause activity. NIST SP 800-115 describes overt testing as less expensive, less risky and more frequently used than covert work, and it notes that staff can treat the engagement as a training opportunity, watching the methods assessors use to evaluate and circumvent security measures.

Who knows: The IT, operations and security staff who run the targets are told the window and scope, so their monitoring reflects a known test rather than an unknown event.
Who approves: Written rules of engagement, using the template in NIST SP 800-115 Appendix B, record the targets, permitted techniques and escalation contacts.
What testers do: They focus on finding and validating vulnerabilities across the agreed targets, with staff available to explain configurations and discuss findings during the test.
Where they stop: The rules of engagement set the limits, and the contact can halt activity if production availability is at risk.
What staff take away: Staff see the attack methods in use, which can shape hardening, monitoring rules and how they triage future alerts.

Before you sign, confirm with the provider how the training element is delivered, such as a debrief session or walkthrough, and whether it is part of the fixed scope or an extra item in the quote.

What overt testing finds and misses, and what covert testing finds and misses

Overt and covert testing answer different questions, so each one shows a different slice of your security posture. The cards below set out what each approach surfaces, what it leaves out, and what to check before you pick one or combine them.

Overt work finds exploitable vulnerabilities

With IT and security staff informed and involved, testers can cover the target list in depth, follow exploit paths to their end, and document each weakness with reproducible steps. NIST SP 800-115 describes penetration testing as using real exploits against production systems and data, which is why overt scope is usually set with named systems and agreed stop points. What it leaves out: overt work tells you little about whether anyone would notice an intruder. Ask the provider to show each finding's exploit path and affected asset.

Overt work builds staff skills

NIST SP 800-115 notes that overt testing can be a training opportunity, because staff watch how assessors evaluate and get around security measures. That knowledge transfers into hardening and detection rules, but only if someone on the team is assigned to capture it. What it leaves out: staff who watch the test know it is happening, so the exercise shows how they respond to a known tester, not how they respond to an intruder. Ask for a walkthrough session as part of the deliverables.

Covert work tests detection and response

Covert testing runs without IT staff knowledge, with upper management's permission, so it examines how the security team responds to perceived incidents. NIST SP 800-115 says it measures the damage an adversary could cause rather than finding every vulnerability. What it leaves out: a covert test that stops at a defined access level will not show the full set of weaknesses behind that boundary. Confirm the agreed boundaries and the escalation path before testing begins.

Covert work exposes technical control gaps

Covert engagements also test whether technical controls such as segmentation, logging coverage and access restrictions behave as designed when an adversary actually moves through them. Gaps surface as attack paths the team had not modeled, not as a list of scanner results. What it leaves out: a covert test rarely produces a complete vulnerability inventory, so pair it with scheduled scanning and overt penetration testing. Ask which controls the engagement will test and what evidence the report will include.

Covert work checks policy knowledge

Covert exercises can reveal whether staff know the policies that govern access requests, data handling and incident escalation, because testers observe what people do when no one has briefed them. This matters most where a documented procedure exists but has never been tested under pressure. What it leaves out: findings about policy knowledge depend on a small sample of staff and scenarios, so they describe behavior on the day rather than the whole organization. Confirm how many staff and processes are in scope before accepting the results.

Covert work costs more to run

NIST SP 800-115 says covert testing is time-consuming and costly because of its stealth requirements, and overt testing is less expensive, carries less risk and is more frequently used. Covert effort is driven by the planning needed to stay hidden, the controls needed to contain the exercise, and the care needed to stop at agreed limits. What it leaves out: a lower price for overt testing does not mean it answers the detection question. Ask each provider to show the planning, containment and reporting effort in the written scope.

What you receive from each approach

This table compares what each approach hands back to the buyer, from rules of engagement and boundary records to staff debriefs and audit evidence. Read it as a checklist of outputs to request, since the exact report format is set in the written scope and should be confirmed with the provider before work starts.

Deliverable	Overt testing	Covert testing
Rules of engagement	Written rules agreed with IT and security staff, naming systems, test windows and contacts for stopping the test	Written rules agreed only with upper management, with a defined stop point such as a certain level of access and a named escalation contact
Scope and boundary records	Target list and exclusions recorded in the scope document, usually shared with the operational team	Boundary record kept by a small control group, showing how far the testers may go before the test stops
Findings report	Vulnerability-level findings with evidence, severity and affected assets, aimed at technical remediation	Narrative of attack paths and the damage an adversary could cause, with fewer but deeper findings and the steps taken
Remediation guidance	Fix-level guidance per finding, which the operations team can act on and retest directly	Priority guidance on control gaps and attack paths, which may need architecture or process changes, not only patches
Staff debrief or training output	Walkthrough of methods, so staff see how assessors evaluate and bypass controls	Limited debrief, since the staff response is itself under test; the buyer should ask how and when the test is disclosed afterward
Detection gap analysis	Limited by design, since staff know the test is running; ask whether logs and alerts are reviewed separately	Core output: which actions were seen, alerted on, or missed, and how the response team reacted to perceived incidents
Evidence for auditors	Scope, methodology and findings report, which can support a penetration testing requirement if the scope meets that standard	Exercise record and boundary documentation, which supports exercise and response review rather than a standard pen test requirement

What happens when a covert test is detected

A detected covert test forces a decision that the rules of engagement should have settled before work started: stop, continue covertly, or continue in the open. The TIBER-EU model offers a useful pattern. In a TIBER-EU test, when the red team is detected, the control team proposes measures to continue covertly. Only as a last resort may testing continue as a limited purple teaming exercise, and that time still counts toward the 12 weeks of active testing in the framework. Ask your provider whether its detection handling follows a similar sequence, and who on your side can approve each step.

Set the detection rules in writing before the first packet is sent. Name the emergency contacts, including a named executive sponsor who holds the authority to stop the test and a technical contact who can confirm within minutes whether activity is the tester or a real adversary. Define stop conditions explicitly: unplanned service impact, access to data outside the agreed boundary, signs that a genuine incident is underway, or a tester exceeding the access level authorized for the engagement. Confirm how a real incident response is handled, since your staff may be responding to the tester as if it were an attacker. Ask the provider to show how it keeps the deconfliction record and how it proves who was acting at any moment.

The debrief should record more than the findings. Capture the time of first detection, which control or person raised it, what the staff did in response, and whether the test continued and under which approval. Note which techniques were caught and which went unnoticed, because that map is the evidence the test was built to produce. Under CIS Controls Safeguard 18.4, validate detections again after the test so they cover the techniques the testers used. Ask the provider to state in its report how detection and stop decisions were handled, so auditors and your board can see the decisions were made under control.

What drives the cost and effort of covert testing

Covert testing costs more than overt testing because the tester must stay hidden while exercising the target. The cards below cover the factors that move the price and effort, who pays for each one, and what to request in a written quote so two proposals can be compared on the same terms.

Stealth requirements

NIST SP 800-115 says covert testing is time-consuming and costly because of its stealth requirements. Every action has to avoid tripping the staff's detection, which adds planning, slower execution and more tester hours than an overt engagement where staff know the test is running. Ask the provider to state how many hours are assumed for evasion and whether the quote covers repeated attempts if the team is detected early.

Size and boundary of the target

Covert work usually stops at defined boundaries, such as a certain level of access, so the target set and the stopping points set the effort. A wider set of facilities, sites or systems adds reconnaissance and tester time. Ask the quote to list the in-scope locations, the access level at which the tester must stop, and what happens if the tester reaches that level sooner than expected.

Defenses and staff roles tested

Covert testing examines technical controls, the staff's response to perceived incidents and policy knowledge. Each added control, monitoring queue or staff role, such as the service desk or on-call engineers, adds scenarios the tester must build and observe. Ask which roles are in scope, which detection tools are expected to fire, and how many separate response paths the engagement will exercise.

Management approval and safeguards

Because IT staff are not told, upper management must approve the work in advance, and the engagement needs agreed safeguards. Legal review, a named contact who can stop the test, and an escalation path for accidental outages all take time from the buyer and the provider. Ask who signs the rules of engagement, who holds the stop authority, and how many review cycles the quote assumes.

Reporting depth

A covert report can describe the damage an adversary could do, the detection gaps and the response timeline, which takes more analysis than a list of vulnerabilities. Deeper reporting, including timelines reconstructed from staff logs and walkthroughs with the response team, adds effort. Ask the quote to specify the report sections, whether a debrief with management is included, and whether staff are interviewed after the test.

When compliance frameworks call for overt or covert work

Most compliance frameworks do not ask for covert testing by name. They require penetration testing and a defined test scope, and the choice between overt and covert work is left to the organization and its assessors, unless a threat-led scheme applies. NIST SP 800-53 Rev. 5 control CA-8 (Penetration Testing) covers the baseline requirement, and its enhancement CA-8(1) describes an independent penetration testing agent or team. CA-8(2) adds red team exercises, which extend penetration test objectives to examine how well an organization's cyber defenses work, and may include technology-based and social engineering-based attacks. SP 800-53B selects CA-8 and CA-8(1) only in the high baseline, so a system categorized at a lower baseline may not carry the red team requirement at all.

Threat-led schemes sit at the covert end of the spectrum. DORA Article 26 requires threat-led penetration testing at least every 3 years for financial entities that their competent authority identifies, and TIBER-EU, in its January 2025 framework, runs threat intelligence-based red team tests in preparation, testing and closure phases with at least 12 weeks of active testing. If a TIBER-EU red team is detected, the control team first proposes measures to continue covertly, and only as a last resort may testing continue as a limited purple teaming exercise that counts toward the 12 weeks. Confirm with your competent authority or regulator whether DORA Article 26 or a national scheme applies to your entity, and whether the scheme expects covert work or accepts an overt exercise.

For the rest of the framework set, CIS Safeguard 18.4 is the practical link to covert work. It asks that security measures be validated after each test so that detections cover the techniques the testers used. An overt test can show a gap in a control, but only a test that goes unannounced to the response team shows whether alerts fired and were acted on. Ask your assessor to map each technique used against your detection coverage and to report which ones were missed. Confirm with your auditor how they will treat an overt test as evidence for Safeguard 18.4 and for any CA-8 finding, since these obligations are judged against your own baseline and regulator.

What does overt vs covert penetration testing cost and require?

Why is covert penetration testing more expensive than overt testing?

Covert testing costs more mainly because of its stealth requirements. NIST SP 800-115 describes covert testing as time-consuming and costly for that reason, while it describes overt testing as less expensive, carrying less risk and more frequently used. The extra cost comes from the tester planning to avoid detection, working within agreed boundaries such as a defined level of access, and documenting what the staff did in response. Ask any provider to price the two approaches separately in the written scope, so you can see which activities drive the difference.

How long does each approach take to run?

NIST SP 800-115 does not give a duration for either approach, so treat any number you hear as a claim to verify. In general terms, overt testing runs on a schedule the staff can plan around, while covert testing takes longer because the tester must stay hidden while working toward defined objectives. Duration also depends on the target, the number of systems in scope and how much remediation is expected in the same window. Ask the provider to state the planned testing window, the reporting period and the point at which the engagement closes, all in the written scope.

What does a covert test need from our staff and management?

A covert test needs a small group of senior people who know it is happening, usually at upper management level, and they must agree the rules of engagement before work starts. The IT and security staff who are not informed still need a clear path to escalate, because their response is part of what the test measures. Management should also decide in advance who can stop the test, how emergencies are handled and who receives the findings. Overt testing asks more of the operations team, who must provide access, answer questions and accept that the test is visible to them.

Can our internal security team run overt penetration tests?

An internal team can run overt tests if it has the skills, time and independence the engagement requires. NIST SP 800-115 treats overt testing as an activity with staff aware and involved, so an internal team can use it for training, with staff observing the methods assessors use to evaluate and circumvent security measures. Independence matters for evidence. Where a regulation or auditor expects an external or independent tester, an internal team will not satisfy that expectation on its own. Confirm the requirement with your auditor or regulator before deciding who does the work.

Does covert penetration testing need regulator approval?

Whether covert testing needs regulator approval depends on your sector and the regulation that applies to you, not on the testing method itself. NIST SP 800-115 describes covert testing as done with upper management's permission, which is the minimum governance it expects. For financial entities in the EU, the TIBER-EU framework runs threat-led red team tests under a control team, and the ECB sets the process. The UK CBEST scheme is run through the Bank of England, which names CREST as the accreditation and certification body. Check with your supervisor whether a scheme applies, and document who authorized the covert work regardless.

How should we combine overt and covert testing?

Most organizations start with overt work and add covert exercises once their controls have been validated. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk. Overt testing finds and fixes known weaknesses so that a later covert exercise does not spend its effort on easy gaps. Covert testing then measures whether detection and response catch the techniques the testers used. Ask the provider how findings from one approach will feed the other, and what will be retested before the covert phase starts.

What should we ask for in a written quote for either approach?

Ask for a quote that separates the cost of planning, testing, reporting and any retest, because each has different drivers. For covert work, request the stealth assumptions, the boundaries where testing stops, the escalation path for staff who detect the tester, and the plan if the team is detected. For overt work, request the access the team needs and who provides it. Ask which costs are one-time and which recur if you repeat the test. Scope, deliverables and timelines are agreed in writing before work starts, so every assumption above should appear in that document.

How DeployOpen scopes, runs and supports overt and covert testing

DeployOpen is a Chainscore Labs company that runs security testing, including penetration testing, inside the client's own infrastructure. Overt and covert work each begin with a written scope that states who knows about the test, what the testers may touch, and where they must stop.

How we scope

Written scope fixes which work is overt and which, if any, is covert, before testing begins
Covert engagements record the senior sponsor who authorized the test and the staff who will not be told
Rules of engagement set access boundaries, stop conditions and escalation contacts for each phase
Engagements are fixed-scope, so deliverables and timelines are agreed in writing before work starts

How we build

Testing runs inside your cloud accounts, data centers or air-gapped networks, not on DeployOpen systems
Test accounts and access paths are provisioned and removed according to the agreed rules of engagement
Covert work is logged with timestamps so detection events can be matched to specific techniques
Testers document each step so the team can separate their activity from genuine attacker activity

How we hand over

Reports separate vulnerability findings from detection and response observations for each test type
Covert findings are shared with the control team and sponsor in a restricted briefing
Remediation guidance is written for the engineers who fix issues and for the staff who respond to alerts
Test evidence is delivered in a form your auditors or regulators can review, without a DeployOpen attestation

How we support

Retesting after fixes confirms whether a finding is closed and whether detections now cover the techniques used
Ongoing support, upgrades and continuous testing are available under a retainer agreed in writing
Ask in any quote which retest rounds, debriefs and evidence packs are included in the fixed scope
Confirm with DeployOpen how a covert test stops if it creates real operational risk

Overt vs covert penetration testing