Threat modeling vs penetration testing: which one your team needs first
Choose threat modeling first when the design is still open, and penetration testing first when a built system is about to ship. Threat modeling analyzes a design for what can go wrong before anything is built. Penetration testing attacks the implemented system to show which weaknesses an attacker can actually exploit. The two answer different questions at different points in the life cycle, so the choice depends on what exists today. The OWASP Threat Modeling Cheat Sheet describes threat modeling as a structured, repeatable process, ideally run early during design and maintained over time, built around four questions: what are we working on, what can go wrong, what are we going to do about it, and did we do a good enough job. Start with threat modeling when a new system, a major architecture decision or a new trust boundary is being designed, such as a new API between a customer portal and a core ledger or a new identity flow. Changes at this stage are cheap, and a model can show where data crosses boundaries and where controls must sit before code exists. Choose penetration testing when a system is implemented or near release and you need exploit-level evidence that a control works. Penetration testing uses real exploits, which NIST SP 800-115 notes carries higher risk than review techniques, so confirm rules of engagement and the environment in writing before any test begins.
For most teams the strongest sequence is both. NIST SSDF task PW.1.1 calls for risk modeling such as threat modeling to help assess security risk, and NIST SA-11(2) guidance warns that systems may deviate significantly from their design, so the model should be updated during development and before delivery. Those updates are what make a later penetration test sharper: each threat the model names becomes a test objective, and each trust boundary becomes a scope item. OWASP's WSTG notes that good threat models do not automatically mean good software, so the model is an input to testing, not a substitute for it. Ask your provider how the threat model will set scope, which assets and abuse cases will be tested, and what evidence will show each threat was addressed. If you must pick one before release, pick the one that matches your gap. A team with no design review yet should model threats first. A team with a stable design and no exploit evidence should test first, then feed the findings back into the model. Confirm with any vendor or assessor exactly which threat modeling method and penetration testing standard they will apply, since NIST does not endorse one methodology over another.
How threat modeling and penetration testing compare on the factors buyers weigh
This table sets threat modeling against penetration testing on the factors that drive scope, staffing and evidence. Read each row as a question a buyer should ask before choosing one, or sequencing both. The main takeaway is that threat modeling examines what could go wrong in a design, while penetration testing proves which weaknesses are exploitable in a built system.
| Factor | Threat modeling | Penetration testing |
|---|---|---|
| Primary goal | Identify threats and decide countermeasures before or during design, following the OWASP Threat Modeling Cheat Sheet's four questions: what we are working on, what can go wrong, what we will do about it, and whether the work was good enough | Validate whether weaknesses in an implemented system can be exploited, following PTES and the OWASP WSTG testing techniques |
| Life-cycle timing | Early in design and maintained over time, which SSDF task PW.1.1 supports as a form of risk modeling | After a build exists, typically before release or on a schedule, as NIST SP 800-115 suggests interspersing with regular scanning |
| Inputs | Architecture diagrams, data flows, trust boundaries, requirements and known adversary profiles | A defined target, rules of engagement, authorized access levels and, under PTES, intelligence gathering and threat modeling of the tested organization |
| Outputs | Threat list, mitigations, design changes and assumptions to verify later | Validated findings with proof of exploitation, severity context and remediation guidance |
| Method at a buyer level | Structured workshops or documented analysis of the design, repeated when the design changes | Hands-on testing of the running system, which PCI SSC describes as essentially a manual process supported by tools |
| Who performs it | Architects, developers and security staff with access to design decisions, sometimes facilitated by an outside party | Independent testers with the skills and authorization to attack, ideally separate from the build team |
| Risk to live systems | None, because no system is touched | Real exploits against production or staging systems and data, which NIST SP 800-115 classes as high risk and requires to be governed by rules of engagement |
| Expected SA-11(2) and PW.1.1 role | Supports SA-11(2) by updating threat models during development and before delivery, since systems may deviate from design | Supports SA-11 penetration testing and the verification of threat model assumptions after the build exists |
| What it misses | Implementation bugs, configuration drift and flaws that the design never anticipated | Flaws in parts of the system outside the agreed scope, and design weaknesses that are not reachable in the tested window |
Why timing decides which approach finds the flaw cheaply
A threat model finds design flaws while they are still cheap to change, and a penetration test finds implementation flaws once something exists to attack. The OWASP Threat Modeling Cheat Sheet describes threat modeling as a structured, repeatable process, ideally performed early in the life cycle during design and then maintained over time. It is built around four questions: what are we working on, what can go wrong, what are we going to do about it, and did we do a good enough job. Answering the second question before the architecture hardens is the cheapest moment to discover that an admin role can reach a payment service directly or that a message queue carries unauthenticated commands. The OWASP Web Security Testing Guide v4.2 (released 3 December 2020) makes the same point at the program level: no single technique is enough, and the recommended mix puts more weight on the early development phases.
Designs do not stay put. NIST SA-11(2) guidance warns that systems may deviate significantly from their design specifications, so updating threat modeling and vulnerability analyses during development and before delivery is critical. For a team, that means the threat model is a living artifact with an owner and a trigger list: a new external integration, a change to authentication, a new data store holding regulated records, or a move to a different hosting zone. Each trigger should force a review of the model before the change ships. Ask your engineering leads which changes currently require a threat model update, who signs off on it, and where the model is stored so a later penetration test tester can read it. If the answer is that nobody updates it after the first release, the model will describe a system that no longer exists, and the penetration test will be the first place the drift shows up.
What threat modeling finds and misses, and what penetration testing finds and misses
Threat modeling and penetration testing look at different objects at different times. The cards below set out what each one surfaces, what it leaves untouched, and how the gaps in one can be covered by the other.
Design-level risks before code exists
Threat modeling examines a design for trust boundary errors, missing controls and abuse paths before a single line is written. It is most useful while changes are still cheap, during architecture review or a new feature's design. Check that the model names each data flow that crosses a trust boundary, and that every identified threat has a recorded decision: mitigate, accept, transfer or avoid.
Implementation defects the model cannot see
A threat model describes what the design intends, so it does not catch a parser that trusts a header or a query built by string concatenation. Those defects need code review, SAST or a test against the running build. Ask whether the model's mitigations are traced to specific code paths, and whether someone verifies that each mitigation was actually built.
Configuration drift after release
Systems often deviate from their design specifications during development and operation, which is why NIST SA-11(2) calls for updating threat models and vulnerability analyses before delivery. A design approved last year may not describe a firewall rule added since. Penetration testing and configuration reviews catch this drift, while the threat model stays silent unless someone revisits it.
Exploitable weaknesses in the running system
Penetration testing uses real exploits against the deployed system, so it shows which weaknesses an attacker can actually chain into access. NIST SP 800-115 describes this as high risk to production, which is why scope, rules of engagement and a staging copy need agreement in writing. Ask the tester to report the attack path, not only the individual finding.
Design flaws the tested paths never reach
A penetration test explores the paths its scope and time allow. A flaw in an administrative workflow, a rarely used batch job or a trust relationship between two internal services may never be exercised. Testers can only sample, so a clean report does not prove the design is sound. Ask which features and roles were excluded and why.
Good models do not guarantee good software
OWASP's Web Security Testing Guide states that good threat models do not automatically mean good software. A model is only as useful as the follow-through on its mitigations and the tests that verify them. Pair each high-priority threat with a named check, either a test case, a code review item or a monitoring rule, and review the model after significant design changes.
When frameworks call for threat modeling, penetration testing or both
NIST SP 800-218 practice PW.1.1 asks teams to use some form of risk modeling, such as threat modeling, attack modeling or attack surface mapping, to help assess the security risk of software. SA-11 in SP 800-53 Rev. 5 covers developer testing and evaluation, and its enhancements name threat modeling, vulnerability analyses and penetration testing as separate activities. The SA-11(2) guidance states that systems may deviate significantly from their design specifications, so updating threat modeling and vulnerability analyses during development and before delivery is critical.
- Design-stage input: PW.1.1 names several forms of risk modeling, so confirm with your assessor whether an attack surface map satisfies the expectation when a full threat model is not feasible.
- Refresh before delivery: SA-11(2) places the emphasis on updating the model during development and again before the system ships.
- Penetration testing is its own activity: SA-11 lists penetration testing as a developer testing enhancement, so a pentest does not replace the design-stage model.
- Who performs the work: SA-11 is written for the developer, so confirm whether the developer or an independent party runs each activity under your contract.
Confirm with your authorizing official or assessor which SA-11 enhancements your system baseline selects, because selection depends on your categorization and tailoring decisions.
CA-8 Penetration Testing sits in the assessment family of SP 800-53 Rev. 5 and has enhancements for an independent penetration testing team, red team exercises and facility penetration testing. In SP 800-53B Release 5.2.0, the baselines select CA-8 and CA-8(1) only in the high baseline, so a moderate or low system does not carry CA-8 as a selected control by default. CA-8 is a testing control, not a design control, so threat modeling needs to be covered by another control such as SA-11.
- Baseline selection: Only the high baseline selects CA-8 and CA-8(1) in the 5.2.0 baselines, which makes the baseline your first check.
- Independence: The enhancement for an independent team is the one to examine if your authorizing official wants separation from the system owner.
- Red team exercises: The CA-8 enhancements mention red team exercises, which are broader than a scoped penetration test and need their own rules of engagement.
- Tailoring: Organizations can tailor baselines, so confirm in writing whether a moderate system adopts CA-8 by agreement.
- Evidence: Ask the assessor which report, scope document and remediation record they expect to see for CA-8.
Before you scope a test as a CA-8 deliverable, check your system categorization and the baseline your authorizing official approved, then confirm the tailoring decisions with that official.
DORA Article 25(1) lists types of ICT testing that financial entities run, including vulnerability assessments and scans, open source analyses, source code reviews where feasible, scenario-based tests and penetration testing. Article 26 requires threat-led penetration testing at least every 3 years for entities that their competent authority identifies. The Article 25 list does not name threat modeling as a test type, so a design-stage model is an input to testing rather than a line item in that list.
- Article 25 is a set of examples: The listed test types are examples, so confirm with your competent authority which mix it expects in your annual testing plan.
- Scenario-based tests need scenarios: A threat model is a practical source for the scenarios your team picks, which keeps the scenarios tied to your actual design.
- Source code review is conditional: The Article 25(1) wording says source code reviews where feasible, so document the feasibility decision and its reasons.
- Article 26 depends on identification: Whether threat-led testing applies to your entity is a decision made by the competent authority, not by the entity alone.
- Tester qualifications: Confirm with the competent authority how testers and providers must be qualified before you contract for any of these tests.
Ask your competent authority which tests it expects in your plan and whether your entity is in scope for threat-led penetration testing under Article 26.
The PCI SSC Information Supplement, Penetration Testing Guidance v1.1 (September 2017), describes penetration testing as essentially a manual process and states that simply running an automated tool does not satisfy the penetration testing requirement. The guidance addresses the test itself rather than design-stage analysis. It does not, in the supplied material, make threat modeling a requirement, so a threat model is best presented as supporting evidence that helps explain the scope of the test.
- Manual work is the test: Automated scanning can feed the engagement, but a tester must perform manual analysis to meet the guidance.
- Design analysis as evidence: A threat model of a payment flow can justify why a particular path was tested, which helps when the assessor questions scope.
- Scope boundaries: Confirm with your QSA which systems and connected paths are in scope before the tester begins, because scope drives both the test and the evidence.
- Version currency: The supplement is v1.1 and dated September 2017, so confirm with your QSA whether that version is the reference they apply.
- Reporting expectations: Ask what the report must show, including how findings were validated and retested.
Confirm with your QSA which tests they expect for your payment environment and whether the v1.1 guidance is the reference they will apply to your assessment.
CIS Controls v8.1 Control 18 covers penetration testing. Safeguards 18.2 and 18.5 ask for external and internal penetration tests at least annually. Safeguard 18.4 asks that security measures be validated after each test so that detections cover the techniques the testers used. Control 18 does not, in the supplied facts, ask for threat modeling, so a threat model is a separate input that a program may add of its own accord.
- Annual cadence: External and internal tests each need to occur at least once a year, so plan them as two tracks with separate scopes.
- Post-test validation: Safeguard 18.4 ties test results to detection coverage, which links the test program to your monitoring and alerting work.
- Operating environment focus: Control 18 targets the operating environment, so design risk is handled through other controls and processes.
- Retest evidence: Keep records showing that detections were checked after each test, since that is what the safeguard asks you to demonstrate.
- Adopted safeguards: Confirm with your CIS assessor which safeguards your program has adopted before you commit to a test cadence.
Check your adopted safeguards against the CIS Controls Assessment Specification for Control 18 before you set the test calendar and the validation steps that follow each test.
How threat model outputs make a penetration test more useful
A threat model is worth the most to a penetration test when its outputs are handed over as test inputs rather than filed as a design document. The abuse cases, data flows, trust boundaries and asset list each map to something a tester can act on. An abuse case such as a customer reading another customer's invoices becomes a named test objective. A data flow that crosses from the public API gateway into the billing service identifies the components that belong in scope. A trust boundary marks where a tester should check that authorization is enforced, not assumed. Assets tell the team which data stores and administrative functions deserve the most attention.
The threat model also decides which roles the tester needs. If the model lists a standard customer, a support agent and a tenant administrator as distinct actors, the engagement should request authenticated accounts for each, with the access those roles really have. Ask the test provider to confirm in writing which of the model's abuse cases become objectives, which are covered by other activities such as code review, and which are out of scope. Keep the threat model version and date in the statement of work so findings can be traced back to the assumptions they tested.
PTES places threat modeling as its own section after intelligence gathering, so a provider following that standard should already be building a model of the tested organization before attacks begin. Ask whether your model will be used as an input or whether the provider will build a separate one. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk. In practice, a design model with a few abuse cases marked for live testing, paired with code review for the authorization logic, usually gives better coverage than a broad test that starts from nothing.
What you receive from each engagement
This table sets the typical outputs of a threat modeling exercise beside those of a penetration test, so you can see what lands on your desk and what you can act on. Read each row as a question to put to a provider, and confirm exact report contents and formats in the written scope before work starts.
| Deliverable | Threat modeling | Penetration testing |
|---|---|---|
| Starting document | Scope of the design reviewed, system diagram and assumptions agreed with the design owners | Scoping and rules of engagement covering targets, testing windows, permitted techniques and emergency contacts |
| Core visual | Diagrams and data flow views showing trust boundaries, data stores and external entities | Test narrative describing the attack path the tester followed, with the systems touched at each step |
| Main output | Threat register listing each threat, affected component, planned mitigation and named owner | Validated findings, each with the evidence captured during exploitation or verification |
| Reproducibility | Assumptions and open questions recorded so the design team can revisit them | Reproduction steps that let your team or a retester confirm each finding |
| Follow-up | Updates to the register as the design changes, and validation that mitigations were built | Retest notes showing which findings are closed, still open or accepted as risk |
| Who acts on it | Architects and developers, who change the design or add controls before build | System owners and remediation teams, who fix the deployed system and track the changes |
| Success check | Validation criteria stating how the team will confirm each mitigation works | Severity and business impact assigned to each validated finding, with the basis for the rating stated |
| Confirm before signing | Required template, register format, diagram tool and whether a sign-off step is included | Report template, severity method, evidence handling, data retention and whether retesting is included |
What drives cost and effort for each approach
Threat modeling cost follows the design work it requires, while penetration testing cost follows how much of the system a tester must reach and how deeply. Read each card as a question to put to a provider or internal team before you accept a quote.
System complexity drives threat model effort
A threat model for a single service with one data store takes far less analysis than one covering dozens of components, third-party integrations and batch jobs. Effort is mostly skilled time spent diagramming data flows and enumerating threats for each element. Ask whether the quote includes a data flow diagram review, which components are in scope, and whether the output is a model a development team can update.
Trust boundaries multiply analysis
Each trust boundary, such as internet to load balancer, service to database or tenant to tenant, creates places where authentication, authorization or data validation can fail. More boundaries mean more threats to record and more mitigations to decide on. Ask how many boundaries the quote assumes and whether each one gets a documented threat and an owner.
Architect and product owner time
Threat modeling depends on people who know the design, including architects, product owners and the engineers who will build the feature. When these people are scarce, sessions run longer or the model reflects assumptions instead of the actual design. Confirm who from your side must attend, how many sessions are planned, and who signs off the final model.
Maintenance keeps the model current
A threat model is only useful while it matches the system. NIST SA-11(2) guidance notes that systems may deviate significantly from their design specifications, so updating the model during development and before delivery is critical. Recurring cost comes from design changes, so ask whether maintenance is included in the quote or sold as a separate retainer.
Target count and access depth
Penetration testing effort scales with the number of hosts, applications and APIs in scope and with how deeply each is exercised. Production access carries higher risk and usually more coordination than staging, which affects scheduling and rules of engagement. NIST SP 800-115 provides a rules of engagement template in Appendix B, so ask whether the quote covers agreed windows, escalation contacts and stop conditions.
Authenticated roles and manual depth
Each user role a tester must log in as adds setup, test cases and access control checks. PCI SSC's Penetration Testing Guidance says penetration testing is essentially a manual process and that simply running an automated tool does not satisfy the requirement, so manual hours are a real cost driver. Ask how many roles are covered, what share of the work is manual and how automated scanning is billed separately.
Questions buyers ask before hiring help for threat modeling or penetration testing
Does a threat model replace a penetration test?
No. A threat model and a penetration test answer different questions, so one does not stand in for the other. A threat model analyzes a design before it is built to ask what can go wrong, while a penetration test attacks the implemented system to see what an attacker can actually exploit. The OWASP Web Security Testing Guide v4.2 makes the point that no single technique is enough and recommends a mix. NIST SP 800-115 similarly notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk. Ask a provider to explain how each activity feeds the other in your scope, not which one you can skip.
How much effort does threat modeling need from my team?
The effort depends on how many people must join the sessions and how much design material already exists. Threat modeling works best with the people who know the architecture, so expect engineering leads and the security owner to take part in the working sessions. The team should provide architecture diagrams, data flow descriptions, trust boundaries, authentication and authorization designs, and a list of external dependencies. The OWASP Threat Modeling Cheat Sheet frames the work around four questions: what are we working on, what can go wrong, what are we going to do about it, and did we do a good enough job. Ask the provider which artifacts they need before the first session and who must attend each one.
How long does a threat model or penetration test take?
No fixed duration applies to every engagement, so treat any single number you see as a claim to verify. Duration depends on the size and complexity of the system, the number of trust boundaries and integrations, how many test environments exist, the access granted to testers, and how quickly your team answers questions and fixes blockers. A threat model usually runs in parallel with design work and is revisited as the design changes. A penetration test has a defined testing window plus time for reporting and retesting. NIST SP 800-115 describes the assessment as a planned technical activity, so ask for a written schedule with milestones in the scope document.
Can my team run threat modeling itself with the OWASP cheat sheet?
Yes, a capable internal team can start threat modeling using the OWASP Threat Modeling Cheat Sheet, which describes the process as structured, repeatable and best done early in design. The practical limits are experience and consistency. Teams new to the method often miss attacker perspectives, skip trust boundary analysis, or fail to record decisions in a way that can be revisited. OWASP WSTG v4.2 lists a practical attacker view and early use in the life cycle as advantages, but also notes that a good threat model does not automatically mean good software. Many organizations run the first few models with an outside facilitator and then move the method in-house, which you can confirm as an option in your own scope.
What does the work need from engineering and security staff?
Engineering provides knowledge of the design and the ability to change it, and security provides the risk framing and the decision authority. Both groups have a role in each phase. Engineers explain how components talk to each other, where data is stored, and what changed since the last design review. Security staff confirm the risk tolerance, the compliance obligations and the findings that must be tracked to closure. NIST SP 800-218 task PW.1.1 treats risk modeling, including threat modeling, as a way to assess software security risk, so the output should feed into engineering backlogs. Name one owner on each side before work starts.
Should we threat model before a penetration test?
A threat model is useful before a penetration test because it sharpens what the tester should attack. The PTES main page places threat modeling after intelligence gathering and treats it as a way for testers to model threats to the tested organization, so the testing phase has clearer targets. A threat model also surfaces trust boundaries and assumptions that a test should check. It does not require a complete design document, but a test scoped without any threat analysis tends to rely on generic checklists. If the system is already built and no model exists, the tester can build a lightweight one as part of the engagement, which you should confirm in the statement of work.
What should I ask for in a fixed-scope quote?
Ask for a written scope that names the systems, environments, trust boundaries, access levels and deliverables before any work starts. DeployOpen agrees scope, deliverables and timelines in writing for most engagements and does not publish a price list, so a quote follows from that document. Ask the provider to list what is one-time, such as the threat model workshops or the initial test, and what recurs, such as retesting or periodic model updates. Ask what drives cost up, including the number of components, the depth of source access, and the number of retest rounds. Also confirm the rules of engagement, the reporting format, who does the work, and what your team must supply in writing.
Who pays for what in a threat model plus pentest engagement?
In a fixed-scope engagement, you pay the provider for the defined work, and your internal staff time is a separate cost that rarely appears on the quote. Provider costs cover facilitation, analysis, testing time, reporting and any retesting in the agreed scope. Your costs include engineer hours in workshops, environment access, test accounts, change freezes and remediation. Combining the two activities can reduce duplicated scoping and knowledge transfer, but only if the same team or a coordinated handoff is planned. Ask the provider to split the quote into the threat modeling and testing phases so you can see which work recurs and which is one-time.
How DeployOpen scopes threat modeling and penetration testing work
DeployOpen plans and runs security testing inside your own infrastructure, whether that is your cloud accounts, data centers or air-gapped networks, so threat models, test traffic and findings stay under your control. Most engagements are fixed-scope, with the scope, deliverables and timelines agreed in writing before any work starts. For a threat model followed by a penetration test, that written scope should name the design artifacts the threat model will cover, the components the test will attack, the accounts and roles the testers will hold, and the rules of engagement that govern production access. Ask for those items explicitly in any quote so that the threat model and the test can be checked against each other.
Ongoing work is handled through retainers, which cover continuing support, upgrades and continuous testing. A retainer is the natural fit when the design keeps changing, because threat models need to be updated as systems drift from their original specifications, a point NIST SA-11(2) makes directly. Confirm with us how a retainer defines the trigger for a threat model refresh, how new findings feed back into the next test scope, and who in your organization approves changes to scope during the term.
DeployOpen prepares systems, controls and evidence for compliance assessments, including the threat models, test records and remediation trails that an assessor will ask to see. DeployOpen does not issue audit reports, attestations or certifications. Those come from independent auditors and certification bodies, so ask your assessor early which artifacts they will accept and in what format. Before signing, confirm the scope, the deliverables, the evidence formats and the timing in writing, and ask for a quote that follows directly from that written scope.
