Choosing a penetration testing provider

Choosing a penetration testing provider starts with what your auditor or customer will accept

Start by asking what your auditor, customer, insurer or regulator will accept as evidence, then choose a provider that can deliver that evidence. For most regulated buyers, the decision turns on three questions: whether the provider performs substantial manual testing, whether its report will satisfy the people who read it, and whether its testers have verifiable skill and experience. Accreditation and certifications help you narrow the field, but they do not replace those checks. Put the acceptance criteria in writing before you request quotes, so every bidder is measured against the same standard.

Provider choice matters because the quality of a penetration test depends on the people doing it. PCI SSC's guidance on penetration testing describes a mainly manual process that exploits vulnerabilities to defeat security features, and it contrasts this with vulnerability scans that run mostly on automated tools in seconds to minutes per host. NIST SP 800-115 likewise frames testing as a planned, documented activity with defined phases, rules and goals. A provider that resells scan output under the label of a penetration test will produce a document that looks complete but does not show whether your controls hold. Ask who will do the work, how much of it is manual, and how the tester's judgment shows up in the findings.

Ask providers for a redacted sample report and check it against the outline PCI SSC suggests, including a statement of methodology, a statement of limitations, traceable severity reasoning and retest guidance. Ask how the provider handles the sensitive access it will receive, such as credentials, production data and network maps, and what happens to that material after the engagement. Confirm the provider's independence from any product or remediation work it might later recommend. Finally, confirm whether the provider's accreditation or certification applies to the individuals doing your test, since PCI SSC notes that certifications alone do not establish experience. Verify these points directly with the provider and, where relevant, with your assessor.

How a buyer evaluates a provider from shortlist to contract

This sequence moves a buyer from a defined objective to a signed, written engagement with a provider whose methods, people, data handling and retest terms have been checked. Each step closes with a document or decision that the next step depends on.

Define objectives and scope from the planning phase
Start by writing down what the test must answer, which systems and accounts are in play, and which compliance requirement or customer question triggered it. NIST SP 800-115 treats planning as the phase where rules are identified, management approval is documented and testing goals are set, and no testing takes place. Your internal sponsor, security lead and system owners should sign off on this draft. The output is a one-page objectives and scope statement that every provider quotes against.
Issue a written request to shortlisted providers
Send the same objectives, scope and report expectations to each provider in a request for proposal, so that quotes are comparable. Ask each provider to state its planned test type, including whether testing will be black-box, grey-box or white-box as defined in the PCI SSC guidance, and why. The output is a set of written proposals that name the people who will do the work, not only the firm that signs the contract.
Review methodology and sample reports
Ask each provider to describe its methodology in enough detail to see where manual work happens and where tools are used. NIST SP 800-115 does not endorse one methodology over another and notes that organizations may adopt or combine them, so judge whether the method fits your scope rather than whether it matches a single named framework. Request a redacted sample report and check it against the PCI SSC report outline, including limitations, severity reasoning and evidence. OWASP ASVS guidance says a test that only runs an automated tool is insufficient, so look for signs of manual validation. The output is a short scoring note on each method and sample.
Check tester qualifications and accreditation
Ask who will actually run the test, what their hands-on experience is with the technology in scope, and how the provider verifies it. PCI SSC lists OSCP, GIAC certifications such as GPEN and GWAPT, CREST penetration testing certifications and CHECK as indicators of skill, and states that it does not validate them and that certifications alone do not establish experience. CREST is a not-for-profit that accredits member companies and certifies individuals; confirm the status of both the firm and the named testers directly with CREST rather than relying on a logo. The output is a named tester list with verifiable credentials and relevant project experience.
Assess independence and data handling
Confirm that the provider does not also build, operate or remediate the systems it will test in a way that compromises its findings, and ask how conflicts are disclosed. Ask where testing data, evidence and credentials are stored, who can access them, how long they are kept and how they are destroyed. Ask what happens if the tester finds data that may be regulated or sensitive. The output is a written answer on independence and a data handling statement that your legal and privacy teams can review.
Confirm retest terms and report ownership
PCI SSC guidance says the tester should retest after remediation to confirm new controls mitigate the original risk, and that remediation completed long after the original test may require a new engagement. Ask whether retests are included in the quote, how many rounds are covered, what is retested and how closed findings are documented. Confirm who owns the report and whether the provider can share it with your auditor or customer. The output is a retest clause and a report ownership clause in the proposal.
Agree rules of engagement in writing before testing
Before any testing, agree the rules of engagement in a signed document that covers approved targets, exclusions, testing windows, source addresses, emergency contacts, stop conditions and the handling of accounts and credentials. NIST SP 800-115 provides a rules of engagement template in Appendix B that you can adapt. Confirm that the statement of work and the rules of engagement match the scope agreed in the first step. The output is a signed contract and rules of engagement, and testing does not start until both are in place.

Which provider qualifications matter most for this test

Certifications are a starting point, not a verdict. The criteria below show how to test a provider's method, people, independence and handling of your data, so you can compare proposals on evidence rather than logos. Ask each provider for written answers before you sign.

Methodology disclosure

Ask the provider to name the methodology it follows and state which parts are manual. OWASP ASVS guidance says testers should disclose their methods in the report and that running an automated tool without thorough testing is insufficient. Ask how long manual testing runs per target, which tools produce the output, and how business logic and authorization are checked. A weak answer is a vague reference to industry best practice, or a description that is mostly scanner output.

Sample report quality

Request a redacted sample report and read it against the PCI SSC outline: executive summary, scope, methodology, limitations, narrative, findings and clean-up. Check that each severity rating explains how it was derived, with traceable reasoning when a CVSS score is adjusted for your environment. Confirm that findings include evidence and remediation steps that your engineers can act on. A weak answer is a sample that lists scanner names and generic findings with no environment-specific reasoning.

Tester experience and certifications

PCI SSC lists OSCP, GIAC GPEN and GWAPT, CREST certifications and CHECK as indications of tester skill, while stating that it does not validate or endorse them and that certifications alone do not establish experience. Ask who will perform your test, how many years they have tested the relevant target type, and whether they have done similar work in your sector. A weak answer is a list of credentials with no named testers and no description of the work they did.

CREST company accreditation

CREST is a not-for-profit company registered in the UK that accredits member companies and certifies individuals. Confirm the provider's current accreditation directly with CREST, and ask which disciplines it covers, since a company may hold accreditation for some services and not others. CREST also announced its first ten companies accredited for AI-Enabled Penetration Testing, so ask whether the provider holds that accreditation if your test covers AI systems. A weak answer is a logo on a website with no accreditation number or scope.

Independence and conflicts

Ask whether the provider sells the remediation, managed security or tooling it will then assess, and whether any of its testers also built or operate your systems. A tester who is independent of the build team is better placed to report on design flaws it helped create. Ask how the provider handles conflicts, who signs off the findings, and whether the report can be reviewed without commercial pressure to soften ratings. A weak answer is that the provider 'works closely with your team' with no written conflict policy.

Safe testing practices

Ask how the provider avoids disrupting production: testing windows, rate limits, backups before intrusive actions, stop-and-call rules for critical findings, and named contacts for emergencies. Confirm that the rules of engagement follow the template in NIST SP 800-115 Appendix B and that your written authorization covers every target, including third-party hosts. A weak answer is that testing is 'non-destructive' with no written procedure for what happens when a test causes an outage.

Data handling and retest terms

Testing exposes credentials, personal data and architecture details, so ask where evidence is stored, who can access it, how long it is kept, how it is encrypted and when it is destroyed. Confirm the retest terms in writing: whether one retest is included, how long after the report it can be requested, and whether a retest is needed after major changes. PCI SSC advises retesting once remediation is complete, and PCI DSS 11.4.4 requires repeating the test to verify corrections. A weak answer is no written retention period or retesting left to a later quote.

How provider evaluation differs for black box, grey box and white box work

Under the PCI SSC definitions, black-box testing means the tester receives no information about the target beyond what is needed to begin. PCI SSC notes that a black-box assessment may require more time, money and resources for the deliverables to meet PCI DSS requirements, because the tester spends effort on discovery that a grey or white box engagement would skip. When you evaluate a provider for this model, the question is whether their proposal accounts for that discovery effort and explains what they will still deliver against your compliance or assurance goal.

Discovery effort: Ask how the provider plans and bills reconnaissance and mapping before any exploitation work begins.
Coverage claims: Check whether they state which assets they expect to reach and which they may not, and why.
Deliverable fit: Confirm the report will still contain the scope, methodology and limitations sections your auditor expects.
Stop conditions: Ask how they handle the moment they find something that needs your immediate attention.
Reasoning on cost: Look for a written explanation of why the black-box price differs from grey or white box work.

Confirm before signing that the proposal states, in writing, whether the black-box deliverables meet the assurance purpose you are buying the test for.

Questions buyers ask when choosing a penetration testing provider

How much does a penetration test cost, and what drives the price?

Price depends mainly on scope size, test type, the depth of manual work, the knowledge level given to the tester (black, grey or white box), retesting, reporting needs and timing. Each provider prices from its own written scope, so two quotes are only comparable when the targets, exclusions and deliverables match. Ask each provider to split its quote into planning, testing, reporting and retest effort, and to state which parts are one-time and which recur. The penetration testing cost guidance on this site walks through each driver in more detail. DeployOpen does not publish a price list; a quote follows from the agreed written scope.

How long does a penetration test take?

PCI SSC's Penetration Testing Guidance describes penetration testing as a mainly manual process that may last days or weeks, depending on scope. Vulnerability scans, by contrast, run in seconds to minutes per host. Duration therefore depends on how many targets, applications and environments are in scope, how much manual exploitation the test requires, and how quickly the client answers questions and provides access. Ask the provider for a schedule that separates planning, testing, reporting and the retest window, and check that the schedule matches the depth of work described in the methodology.

What must the buyer provide before a penetration test starts?

The buyer should provide the targets, the business purpose of the test, the accounts and roles the tester needs, any exclusions, the testing window and named technical contacts. NIST SP 800-115 treats planning as the phase where rules are identified, management approval is documented and goals are set, and no testing occurs until then. Expect to sign rules of engagement that cover emergency contacts, data handling and what happens if a critical issue appears mid-test. Ask the provider which of these items it will need in writing before work begins.

Can our internal security team run the penetration test?

An internal team can perform useful security testing, but the test is weaker evidence when the same people built or operate the systems being tested. Independence matters for compliance and customer reviews, and PCI DSS expects testing to be performed by a qualified internal resource or an external firm that is organizationally separate from the team that manages the environment. Confirm the rule your framework applies, then ask whether the internal team can show documented methodology, the report outline it will produce and its retest process. Many organizations use internal staff for continuous checks and an external provider for the formal annual test.

Is a certification enough to choose a provider?

No. PCI SSC lists OSCP, GIAC certifications such as GPEN and GWAPT, CREST penetration testing certifications and CHECK as indications of tester skill, but it states that it does not validate or endorse them and that certifications alone do not establish experience. Ask for named testers' recent project history, a sample report with the methodology statement, the tools used and how findings are ranked. CREST also accredits member companies, which is a company-level signal that sits alongside individual credentials. Check accreditation status directly with the accrediting body, since it can change.

Does an AI-enabled penetration test need CREST's AI-enabled accreditation?

CREST announced its first ten companies accredited for AI-Enabled Penetration Testing in September 2026, so accreditation for that discipline now exists. Whether you require it depends on your framework, your customer and what the test covers. If a provider uses AI tooling to assist manual work on a conventional application, ask how the tooling is governed and whether the report discloses it. If the test targets AI systems themselves, such as LLM applications, RAG pipelines or agents, ask how the provider tests against OWASP GenAI guidance and MITRE ATLAS. Confirm the current accreditation list on CREST's site before you rely on it in procurement.

What should we do if a provider only runs automated scans?

Do not accept a scan as a penetration test. A vulnerability scan identifies and ranks issues mostly through automated tools, while a penetration test is a mainly manual process that attempts to exploit vulnerabilities to defeat security features. OWASP ASVS notes that running an automated tool without thorough testing is insufficient. Ask the provider to describe which steps are manual, which testers performed them, and how logic and authorization flaws were tested. If the answer is mainly tool output, request a scan under its own label, or look for a different provider for the penetration test itself.

What a provider should deliver and whether it satisfies an auditor

A penetration test is only as useful as the evidence it leaves behind. The cards below cover what a provider's report should contain, how severity should be justified, how retests close the loop, and where the provider's role ends and an auditor's begins.

Report structure to expect

The PCI SSC suggested outline includes an executive summary, statement of scope, statement of methodology, statement of limitations, testing narrative, segmentation results, findings, tools used and environment clean-up. Ask for a sample report before signing and check that each section is present. A report missing scope or limitations cannot show what was and was not tested, which is the first question an assessor will ask.

Findings referenced to CVE and CWE

Each finding should cite the relevant CVE identifier where one exists and the CWE weakness category where it does not, so reviewers can match issues to known vulnerabilities and recurring weakness classes. Check that findings describe the affected component, the condition observed and the evidence captured, not only a generic title copied from a tool.

Severity traceable to CVSS

Severity ratings should state how they were derived, typically from CVSS base scores. If the provider adjusts a score for your environment, the report should show the reasoning, such as compensating controls or exposure, so your team can challenge it. Unexplained labels like critical or high give your risk owners nothing to act on or defend.

Retest results and closure evidence

Confirm in the contract that the provider retests after remediation and documents each original finding as closed, open or partially mitigated, with the evidence used. PCI DSS requirement 11.4.4 expects exploitable issues to be corrected and the testing repeated to verify the corrections. Ask how long after the original report retests remain covered, since long delays may call for a new engagement.

Attestation letter the buyer can share

Many providers issue a letter confirming that a test was performed, its dates and its scope. Ask what this letter says and what it does not say. It is a statement of work performed, not an audit report or certification. Your assessor or customer will still need the full report and may request the methodology, tester qualifications and limitations alongside it.

Where DeployOpen fits the audit

DeployOpen runs security testing and prepares systems, controls and evidence for compliance assessments. It does not issue audit reports, attestations or certifications. Independent auditors and certification bodies make those determinations. Before commissioning a test, confirm with your assessor which report format, tester evidence and retest records they will accept for your framework.

Which frameworks set expectations for the testing provider

This table compares the frameworks that most often shape a penetration test and what each expects from the test or the provider running it. Read each row as a question to put to a provider: how often the test must happen, what happens to findings, and what evidence an assessor will ask to see.

Framework and control	What it expects of the test or provider	Where buyers usually get caught
PCI DSS 11.4.2 and 11.4.3	Internal and external penetration tests at least every 12 months and after significant changes, performed by a qualified internal resource or external firm	Testing is booked once a year but is not repeated after a major platform or network change
PCI DSS 11.4.4	Exploitable vulnerabilities are corrected according to the entity's risk ranking, and the testing is repeated to verify the corrections	The provider delivers a report but no retest is in the contract, so closure cannot be evidenced
CIS Controls Safeguard 18.2	External penetration tests at least annually, with the assessment failing if more than twelve months pass since the last external test	Teams run an internal scan and count it as an external test
CIS Controls Safeguard 18.5	Internal penetration tests at least annually, covering the internal network and its trust relationships	The internal test is scoped too narrowly to reflect where the client's real risk sits
OWASP ASVS 5.0.0	The testing method is disclosed in the report, and running an automated tool without thorough manual testing is insufficient	The provider's sample report shows scanner output with little manual analysis behind it
DORA threat-led testing (SWD(2020) 199)	The European Commission's impact assessment cites an ESA estimate that threat-led testing costs 0.1% to 0.3% of the total ICT budget of the firms concerned	Buyers treat the cost estimate as a price benchmark, when it reflects an ICT budget share and not a quote
Provider qualifications (PCI SSC guidance)	Skill indicators include OSCP, GIAC GPEN and GWAPT, CREST certifications and CHECK, but PCI SSC does not validate them and certifications alone do not prove experience	Certificates are accepted as proof of competence without asking who will run the test and what they have done

Red flags that a provider sells scanning as penetration testing

The most common buying mistake is paying for a penetration test and receiving a vulnerability scan. The two are different services. PCI SSC's guidance describes vulnerability scans as mostly automated, identifying and ranking weaknesses in seconds to minutes per host, while a penetration test is a mainly manual process in which a tester tries to exploit weaknesses to defeat security features, and which may last days or weeks depending on scope. A provider that quotes a short, low-cost engagement for a large estate, or that delivers results within hours, may be selling a scan under another name. Ask how many hours of manual testing the quote includes and who will perform them.

Several warning signs appear during the sales process and in the sample report. The first is no disclosed methodology. OWASP's ASVS guidance says certifying organizations should disclose their testing methods in the report and that simply running an automated tool without thorough testing is insufficient, so a provider that will not name its methodology or map its work to a recognized framework has given you no basis for judging it. The second is a report that reads like a scanner export: a long list of CVE matches with no narrative, no attack paths, no validation of exploitability and no explanation of how severity was derived. The third is no named testers. Ask who will do the work, what they have done before and how they are supervised. Certifications such as OSCP or CREST's individual certifications are indications of skill, not proof of experience, and PCI SSC states they do not establish it on their own.

The last group of warning signs concerns terms. Be wary of vague limitations, such as a statement of limitations that says only that testing is "best effort" or omits the systems excluded and the techniques not attempted. Refusal to retest is another red flag, because PCI SSC expects a tester to retest once remediation is complete. Finally, no written rules of engagement should end the conversation. NIST SP 800-115 treats the rules of engagement and documented management approval as part of planning, before any testing occurs. A provider that starts testing without them, or treats them as a formality, is not practicing the discipline your auditor or customer will expect.

Choosing a penetration testing provider