Penetration testing vs red teaming

Penetration testing vs red teaming: which one your organization needs first

Penetration testing gives you coverage of defined targets, while red teaming gives you objective-driven adversary emulation that also tests detection and response. If you need one answer first, choose penetration testing unless your exposed systems are already well hardened and your open question is whether your security operations team would notice a determined intruder. That decision is really a sequencing question. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk, so the practical choices are which approach comes first, how long the gap between them runs, and who owns each result. Ask each provider to state in writing which of the two they are actually selling before you compare quotes.

A first red team usually measures less than it appears to when the baseline is unknown. A penetration test with a written scope and rules of engagement tells you which applications, hosts and network segments can be exploited, what it took to do so, and gives your engineers a concrete list to fix. If a red team reaches its objective without that baseline, you cannot tell whether the path used a known and unpatched weakness, a misconfiguration or a detection failure. CIS Safeguard 18.4 asks for security measures to be validated after each test so that detections cover the techniques the testers used, which only works when those techniques are documented first. Use the rules of engagement template in Appendix B of NIST SP 800-115 to set the terms of the penetration test before you commit to any adversary simulation.

Add a red team when the question changes from whether weaknesses exist to whether your people and tools detect and respond to an intruder working toward business goals. *NIST SP 800-53* Rev. 5 describes red team exercises as simulations of attempts to compromise mission and business functions, which is a different test from a scoped list of targets. In regulated sectors the trigger may be external. DORA Article 26 requires threat-led penetration testing at least every 3 years for entities their competent authority identifies, and TIBER-EU schedules 4 to 6 weeks of threat intelligence and 2 to 3 weeks of planning before at least 12 weeks of active testing. Confirm with your regulator whether either applies to you, and confirm with each provider whether their durations and purple teaming work fit your calendar.

How penetration testing and red teaming differ on scope, team, rules and reporting

This table compares the two approaches on goal, scope, duration, team, rules, overtness, reporting and whether detection and response are measured. Read each row as a condition: the advantage on one side usually holds only when the stated circumstances apply. The main takeaway is that a penetration test answers whether known weaknesses can be exploited, while a red team answers whether the organization notices and responds to a determined adversary.

Dimension	Penetration test	Red team exercise
Primary goal	Find and validate weaknesses across defined targets so they can be fixed. Best when the control gaps are not yet known.	Test whether an adversary can reach a stated objective and whether defenders detect and respond. Best when basic coverage is already verified.
Scope	Defined systems, applications or networks listed in advance. Easier to budget and to repeat on a schedule.	Objectives and business functions set by the client, with the route left to the team. Requires strict boundaries on systems that must not be disrupted.
Duration	Fixed, bounded engagement agreed in writing before work starts. Confirm the exact length in the scope document.	Longer, covert activity against a live environment. NIST SP 800-115 treats covert testing as a distinct, more involved choice. Confirm length with the provider.
Overtness	Usually overt, so IT and security staff know testing is under way. NIST SP 800-115 describes overt testing as less expensive, lower risk and more frequently used.	Often covert, run without the IT staff's knowledge but with upper management's permission. Its value depends on a sponsor who can stop the test if it causes harm.
Team	Testers with broad technical skills working against the listed targets. Confirm named testers and their qualifications before signing.	A team with roles such as lead and operators. The Bank of England's CBEST guide names CREST Certified Red Team Manager, CREST Certified Red Team Specialist and Cyber Scheme Red Team Manager as qualifications for red team roles.
Rules of engagement	Set out targets, testing windows, permitted techniques, data handling and escalation contacts. NIST SP 800-115 provides a rules of engagement template in Appendix B.	Needs the same rules plus explicit limits on deception, social engineering, physical access and stop conditions. Confirm the escalation path for a live incident.
Reporting	Lists validated findings, exploitation evidence and remediation guidance, and feeds a fix-and-retest cycle. Most useful when the output is a prioritized fix list.	Narrates the attack path, the objectives reached and where defenders detected or missed activity. Most useful when the output is a detection and response improvement plan.
Measures detection and response	Not the primary goal. Ask the provider whether detection is recorded at all, since a tester who is noticed may end the work early.	Yes, this is a core output. TIBER-EU threat-led tests produce no pass or fail result, so the value lies in lessons learned rather than a score.

What each approach is trying to prove

A penetration test asks how many exploitable weaknesses it can find and validate inside the agreed scope. The Penetration Testing Execution Standard organizes that work into seven phases, from pre-engagement interactions through reporting, and the OWASP Web Security Testing Guide v4.2 adds that no single technique is enough, so testers combine manual inspection, threat modeling, source code review and penetration testing. The result is a finding list tied to named hosts, applications and roles. Before you sign, confirm in the statement of work which targets, accounts and attack paths are in scope, because the tester can only prove what the scope allows.

A *red team exercise* asks a different question: could an adversary reach a business objective without being stopped? NIST SP 800-53 Rev. 5 control CA-8 separates the two, and its guidance for enhancement CA-8(2) describes red team exercises as simulations of attempts to compromise mission and business functions that reflect real-world conditions, while penetration testing may be primarily laboratory-based. The objective, such as reaching a payment system or exfiltrating a dataset, sets the success criteria, and the defenders are usually not told in advance. Ask the provider how it will define the objective and whether the rules of engagement allow it to pivot through trusted systems.

NIST SP 800-115 describes covert testing as work done without the IT staff's knowledge but with upper management's permission, and it says that kind of test examines the damage an adversary could cause rather than trying to find every vulnerability. Overt testing, by contrast, is less expensive, carries less risk and is used more often. Decide early who approved the covert work, who the emergency contact is, and what happens if the team is discovered, since those answers belong in the rules of engagement. If your goal is a complete vulnerability inventory, a covert exercise will leave gaps by design.

What each approach finds and misses

Penetration testing and red teaming answer different questions, so each leaves different gaps. The cards below set out what each approach surfaces, what it leaves untested, and what the buyer must check before treating the result as proof of detection or resilience.

Validated weaknesses inside scope

A penetration test uses real exploits against the systems the client names in scope and reports only what the tester could actually validate. It finds exploitable weaknesses in those defined targets, which makes its findings concrete enough to prioritize and retest. Its blind spots are anything outside the written scope, including assets nobody listed, third-party systems and chained attack paths that cross into unscoped networks. Check the scope document for exclusions before reading a clean report as a clean environment.

Attack paths across people and process

A red team engagement pursues a defined objective, such as reaching a business-critical system, and may combine phishing, social engineering, credential misuse and technical exploitation. It finds attack paths that cross people, process and technology, which a target-by-target test seldom assembles. Its trade-off is breadth: a red team typically samples a few routes to the objective rather than cataloging every vulnerability, so a clean objective result does not mean the estate is free of exploitable flaws.

Detection and response coverage

Red teams are designed to test whether the security operations team notices activity and responds, and the outcome is recorded as part of the exercise. Penetration testing reports rarely measure detection, because testers often work openly with the client or announce their presence. Ask whether the engagement is overt or covert, whether the blue team was informed, and who records when activity was first noticed. Without those answers, neither result shows detection coverage.

Validating detections after a test

CIS Safeguard 18.4 asks that security measures be validated after each test so that detections cover the techniques the testers used. This is the step that turns either exercise into detection improvement. For a penetration test, map each finding to the log sources and alerts that should have fired. For a red team, replay the executed techniques against the monitoring stack and confirm which ones produced alerts, which were logged without alerting, and which were invisible.

The question a pentest cannot answer

A penetration test cannot tell you whether anyone would have noticed the attack. Its report describes what was exploitable, not whether a security operations team detected, triaged or contained it during the test window. Only an exercise designed with detection as a measured objective can answer that question, and even then the answer depends on rules of engagement, test duration and whether the defenders were informed. Ask the provider to state in writing what the test does and does not measure.

Regulatory scope and evidence

TIBER-EU red team tests produce no pass or fail result, and DORA Article 26 requires threat-led penetration testing at least every 3 years for entities identified by their competent authority. Those schemes define what evidence is expected, which is different from a standard penetration test report. Confirm with your regulator or auditor whether a required test must be threat-led, and whether a conventional penetration test satisfies the requirement or only supports it.

How to sequence penetration tests and a first red team

The sequence builds a measured baseline, confirms fixes hold, and proves detection works before an objective-driven red team tests the whole organization against a realistic adversary.

Establish a baseline with scanning and periodic penetration tests
Start with regularly scheduled network and vulnerability scanning, interspersed with periodic penetration testing, as NIST SP 800-115 suggests. The internal security team or a third-party tester runs coverage-driven tests against defined targets with written rules of engagement. The output is a findings register ranked by business impact, plus a clear record of which systems, applications and access levels were covered. The step is done when every in-scope target has a report and the gaps in coverage are documented.
Validate that remediation actually closes findings
Owners fix the findings from the baseline, and the testers retest each one against the original evidence. Retesting confirms that a fix works and does not introduce a new weakness nearby. The output is a retest report that marks each finding as closed, partially fixed or still open, with the reason recorded for any risk accepted. The step is done when no critical or high finding remains open without a signed risk acceptance.
Validate detections against the techniques testers used
Under CIS Safeguard 18.4, security measures should be validated after each test so that detections cover the techniques the testers used. The monitoring team checks whether logs were produced, whether alerts fired and whether analysts responded to the activity from the penetration test. Gaps become tuning or detection engineering tickets. The output is a detection coverage list for each test technique. The step is done when every technique used in the test has been seen by monitoring, or has a documented gap and an owner.
Scope a first red team with rules of engagement
Once the baseline is stable and detections are verified, define an objective-driven exercise. The sponsor, usually senior management, sets the business objectives, the adversary profile and the boundaries, including what is off limits, how testers escalate, and who can stop the exercise. NIST SP 800-115 provides a rules of engagement template in Appendix B that can anchor the document. The output is a signed rules of engagement, a limited number of named contacts and a clear decision on whether the exercise is overt or covert. The step is done when the sponsor and the legal and operations leads have signed off.
Run the red team and record what defenders saw
The red team emulates an adversary toward the agreed objectives, while the defenders work without knowing the activity is a test unless the design says otherwise. The team logs each action, time and target so the defenders' records can be compared later. The output is a timeline that links attacker actions to what the security operations team detected, missed or escalated. The step is done when the objectives have been attempted or the sponsor has stopped the exercise, and the team has handed over its evidence.
Close the loop with purple teaming and a joint debrief
Attackers and defenders then work together on the missed or weak techniques, replaying them and tuning detections and response playbooks. TIBER-EU makes purple teaming mandatory within its own regulated scheme, and it is a useful follow-up for any first red team. The output is a set of verified detection changes and a debrief report that separates control failures from process failures. The step is done when each missed technique has been replayed against the updated controls and the results are recorded.
Confirm readiness before commissioning the first red team
Before signing, confirm that baseline findings are closed or formally accepted, that detections have been validated against earlier test techniques, and that a named owner can stop the exercise. Ask the provider which certified staff will lead, and check the qualifications they hold against CREST or the CBEST guide where relevant. Ask for the duration, the methods, the reporting format and the handling of any real sensitive data in a quote. The step is done when these answers are written into the engagement letter.

What rules of engagement and safety controls change between the two

The rules of engagement are where a penetration test and a red team exercise diverge most in practice, and the buyer sets most of those differences before any testing starts. NIST SP 800-115 provides a rules of engagement template in Appendix B, and it is the natural starting point for both formats. A standard penetration test typically needs a signed scope, named target systems, permitted techniques, testing windows, and a clear contact list for stopping work. A covert red team needs considerably more. Decide in advance who in the organization knows the exercise is running, which people hold that knowledge (usually a small group in senior management), and how that knowledge is protected so it does not leak to the staff whose detection is being measured.

A covert red team also needs tighter control over escalation and stop conditions. Define in writing what triggers a pause, such as a live service impact, discovery of a genuine intrusion by a different actor, or access to data outside the agreed set, and who must be called at each step. Decide how the provider handles real data it reaches, including whether it may copy, exfiltrate or only prove access, and where any evidence is stored and destroyed. Ask the provider to confirm how its operators are vetted, how they are supervised during the exercise and what happens if a tester is identified by the defenders. Since the exercise may be mistaken for a real attack, also agree how the deconfliction process works with the security operations team and any managed service providers.

PTES describes pre-engagement interactions as the stage where the agreement is shaped, covering scope, goals, rules of engagement and communication, so use that stage to settle these points before a contract is signed. For a penetration test, check that the provider can show its rules of engagement template, its escalation contacts and its handling of test data. For a red team, ask whether the provider's approach follows the guidance in the TIBER-EU framework or the CBEST guide where those apply, and what the regulator or sponsor must approve. Confirm the exact formats, named qualifications of the lead team and the review points with the provider in writing. Those answers will tell you more about the safety controls than any general description on a website.

Which approach fits which buyer situation

A team about to expose a new application or network for the first time usually needs to know what an outsider can reach and which weaknesses can be exploited. A penetration test with a defined scope answers that question. NIST SP 800-115 treats penetration testing as a validation technique that uses real exploits, so the rules of engagement should set limits before any testing starts. Illustrative example: a team launching a customer portal would normally begin with a scoped web application test before considering anything covert.

What it answers: Which known weaknesses can be exploited against named hosts, services or application functions, and how serious the consequences would be.
Scope: Named hosts, URLs, APIs and test accounts, plus any shared or third-party systems that are excluded and why.
Rules of engagement: Testing windows, emergency contacts, data handling and stop conditions, using the template in NIST SP 800-115 Appendix B as a starting point.
Method: Confirm whether the provider follows PTES or the OWASP Web Security Testing Guide, and whether the work is manual, automated or both.
Retest: Ask how fixes are verified and whether retesting is included in the fixed-scope agreement.

Before buying, confirm the scope in writing, the names and experience of the testers, and whether the report gives developers enough evidence to reproduce each finding.

What drives cost and effort for each approach

Penetration tests and red team exercises are both priced from a written scope, but the drivers differ in kind. A penetration test scales with the number of targets and how deeply each is exploited, while a red team exercise scales with objectives, covert operations and the time the team spends building an adversary-style campaign.

Scope size and target count

A penetration test is usually priced by the number of hosts, applications, APIs or sites in scope and how many roles each application needs tested. A red team exercise is scoped around a few business objectives instead, so the number of systems matters less than how many paths the team must investigate to reach them. Ask the provider to state which assets are in scope, which are excluded, and how additional targets found during testing are handled.

Depth of exploitation

Penetration testers validate each finding with a controlled exploit, and deeper exploitation takes more hours per target. Red team operators go further, chaining access, moving laterally and maintaining footholds toward the stated objective. Ask whether post-exploitation, persistence and data-access proof are included, and whether a stop point is set for each exercise.

Covert operations and rules of engagement

Covert work, where IT staff do not know a test is running, adds planning for cover stories, deconfliction and safe stop procedures. Rules of engagement set the hours, permitted techniques, excluded systems and escalation contacts, and negotiating them takes senior time on both sides. Once agreed, these terms are largely fixed for the engagement, so changes mid-exercise usually add cost. Confirm who approves covert actions and how an accidental outage is handled.

Team qualifications and staffing

Red team roles carry premium rates because the bar is higher. The Bank of England's CBEST implementation guide names the CREST Certified Red Team Manager (CCRTM), CREST Certified Red Team Specialist (CCRTS) and Cyber Scheme Red Team Manager (CSRTM) qualifications for these roles. Ask who will lead the team, which of these qualifications they hold, and whether the same people deliver the closing report.

Reporting and retesting

Penetration test reports list findings with remediation guidance, and retesting of fixed items is commonly quoted separately. Red team reporting covers the attack narrative, which detections fired and which did not, and a debrief. Both produce one-time deliverables, but repeat engagements recur. Ask what retesting is included, whether the report maps findings to detection gaps, and what happens if the team must return after fixes.

Buyer effort and threat intelligence

The buyer's own staff carry a real share of the work: naming objectives, providing accounts and access, coordinating with operations and running the remediation backlog. Threat-led exercises add a preparation phase in which threat intelligence informs the scenario. The TIBER-EU framework (ECB, January 2025) indicates 4 to 6 weeks of threat intelligence and 2 to 3 weeks of red team test planning before at least 12 weeks of active red team testing. Budget internal time for those phases before any quote is accepted.

Questions buyers ask before choosing a penetration test or red team

Does a red team replace penetration testing?

No. A red team exercise supplements penetration testing and does not replace it. NIST SP 800-115 says organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk, and it suggests regularly scheduled scanning interspersed with periodic penetration testing. NIST SP 800-53 Rev. 5 control CA-8 Penetration Testing lists red team exercises as an enhancement on top of the base control, not as a substitute for it. Penetration testing checks defined targets for weaknesses, while a red team tests whether an adversary can reach an objective and whether your staff detect and respond. Most organizations run penetration tests first, then add a red team once their known weaknesses are under control.

How long does a red team engagement take?

Duration depends on the exercise format and the regulatory scheme, so ask the provider for a written timeline tied to the scope. The only minimums in the supplied research come from TIBER-EU, the European Central Bank's framework. It indicates 4 to 6 weeks of threat intelligence work and 2 to 3 weeks of red team test planning before at least 12 weeks of active red team testing. Those figures apply to TIBER-EU engagements, which are intended for regulated financial entities. A custom red team for a non-regulated organization may run shorter or longer, and you should confirm the planning, active testing and reporting phases in writing before work starts.

What does a red team or penetration test cost?

No public figure is supplied for this comparison, so ask for a quote built from a written scope. The main cost drivers are the number of targets, the depth of the objective, how long the team is active, whether social engineering or physical access is included, how much threat intelligence the provider must produce, and how many people with specialist qualifications the work needs. Red teams cost more than penetration tests mainly because they involve more planning, more covert effort and more reporting time. Ask the provider to split the quote into planning, testing, reporting and any retest, and to state which items are one-time and which recur when you repeat the exercise. DeployOpen does not publish a price list. A quote follows from the agreed scope.

What does a red team need from our own team?

A red team needs a named internal owner, a small set of trusted contacts who know the exercise exists, and a clear way to stop the test if it threatens live services. NIST SP 800-115 describes covert testing as work done without the IT staff's knowledge but with upper management's permission, so senior sponsorship must be documented before the start. You will also need to provide architecture information, asset inventories, and access to people who can explain business functions the adversary might target. In TIBER-EU engagements, the regulated entity works with a control team and the purple teaming phase requires your defenders to take part. Confirm with the provider exactly which staff are informed, who holds the control group role, and how escalation works.

Can our own staff run penetration tests or red team exercises in house?

Your staff can run some penetration tests in house, but whether that satisfies a requirement depends on the standard or regulator. NIST SP 800-53 control CA-8 has an enhancement for an independent penetration testing agent or team, so independence is part of what the control asks for. DORA Article 26 requires threat-led penetration testing for entities their competent authority identifies, and TIBER-EU requires testers and the threat intelligence provider to meet defined qualifications. Before deciding, confirm with your auditor, regulator or assessor whether independence is required, and whether the team holds the qualifications the scheme expects. The Bank of England CBEST guide names CREST and Cyber Scheme red team qualifications for red team roles, which is worth checking against your own staff.

What does a covert red team test reveal if the team gets caught?

A detection is a result, not a failure of the exercise. NIST SP 800-115 says covert testing examines the damage an adversary could cause rather than trying to find every vulnerability, so the outcome you care about is how far the team got before a defender noticed and what the response looked like. Ask the provider in advance whether the test pauses after detection, whether the blue team is told, and whether testing resumes to reach the objective. Record the timeline of each action and the detection gap it reveals, because that is the material that improves monitoring. Any exercise should include a stop procedure in the rules of engagement, so detection does not turn into an unplanned production incident.

Does a red team report give a pass or fail result?

No. The European Central Bank states that TIBER-EU tests produce no pass or fail result, and red team reports are generally not scored that way. A red team report describes the objective the team pursued, the attack paths used, the controls that worked, the detections that fired or failed, and the findings that need remediation. Ask the provider for a report structure before the engagement, and check whether it covers the rules of engagement, the threat intelligence used and the purple teaming output. Treat it as evidence for decisions about detection and response, not as a certificate of security.

What you receive from each engagement and how to use it

A penetration test produces a report of validated findings: each weakness tested with a real exploit or a documented proof of exploitation, its affected asset, its severity and a remediation path. The part most buyers overlook is the retest. Ask the provider for a retest record that shows, for each finding, the date it was rechecked and the outcome. NIST SP 800-115 describes penetration testing as using real exploits against production systems and data, so the report should also state exactly which hosts, accounts and dates were touched. Those details let an assessor trace each finding to a fix. A red team report looks different. Its core is an attack narrative: the initial access route, the steps taken toward the agreed objective, and the point where the objective was reached or blocked. It also lists detection gaps, meaning the actions that went unseen by the security operations team, and the moments when an alert fired and how the response unfolded.

Use these outputs for different audiences. Penetration test findings feed remediation tickets and can serve as technical evidence for a control assessor that asks whether testing took place and whether results were tracked to closure. A red team report is better suited to executive and board discussion about whether an adversary could reach a critical function, and to the detection and response team's improvement plan. Neither report is an audit report. DeployOpen prepares evidence, such as the scope letter, rules of engagement, reports, retest records and remediation logs, but does not issue audit reports, attestations or certifications. Independent auditors and certification bodies decide what the evidence proves. Before the engagement starts, ask the auditor or regulator which artifacts they will accept and in what format, and write those requirements into the statement of work.

The most useful red team deliverable is the debrief, not the report alone. A purple team session or structured debrief walks the defenders through each step the testers took, maps it to the detections that should have fired, and records the tuning changes, new detection rules and response playbook updates that followed. Require that the debrief produce a dated action list with named owners. Then schedule a validation run that repeats the techniques that were missed, so the security team can confirm that detections now cover them. CIS Controls v8.1 Safeguard 18.4 asks for this kind of post-test validation. For regulated entities, this closed loop is often the strongest evidence of improvement, because it shows the same technique was tested, missed, fixed and checked again.

How DeployOpen scopes, runs and supports penetration tests and red team exercises

DeployOpen plans, runs and documents penetration tests and red team style exercises inside the client's own cloud accounts, data centers or air-gapped networks. Engagements are fixed-scope, with objectives, rules of engagement and deliverables agreed in writing before any testing starts.

How we scope

The written scope names targets, viewpoints, access levels and whether testing is overt or covert.
Rules of engagement cover testing windows, permitted techniques, escalation contacts and stop conditions.
Penetration tests and red team exercises are priced from the written scope, so ask for each assumption in writing.

How we build

Test accounts, environments and monitoring access are provisioned and checked before the first test step.
Testers follow a documented methodology and record evidence as they go, so findings can be reproduced.
For red team exercises, the team agrees which detection and response controls will be observed and how they will be measured.

How we hand over

Reports separate validated findings from observations, with reproduction steps, affected assets and recommended fixes.
Red team deliverables describe the objectives reached, the detection gaps seen and the response actions taken.
Retest of agreed findings is documented so the client can show remediation status to its own auditors or regulator.

How we support

Ongoing support, upgrades and continuous testing are available as retainers after the initial engagement.
Retainers can cover scheduled penetration tests, retesting and changes to scope as systems and threats evolve.
Ask DeployOpen to confirm which retainer terms, response expectations and reporting cadence apply to your engagement.

Penetration testing vs red teaming