Manual vs automated penetration testing

When manual vs automated penetration testing fits your scope

If your shortlist includes both approaches and an audit or compliance deadline is close, the first question is which evidence the assessor will accept. PCI SSC's penetration testing guidance describes penetration testing as essentially a manual process, and says simply running an automated tool does not satisfy the requirement. OWASP ASVS 5.0.0 takes the same line: automated DAST and SAST tools cannot fully verify many requirements, especially business logic and access control. Automated scanning sits between manual tests, as the logical first step in a network-layer test and as a regular check in the months between tester-led exploitation. Confirm with your assessor which method and report depth they expect before you sign anything.

Make tester-led work the primary choice when your riskiest exposure is in logic, not in known signatures. That covers custom web applications and APIs with multiple roles, workflows that move money or approve access, and any environment where an authenticated user can reach data belonging to someone else. An automated tool can miss a hidden administrative parameter or a home-built authentication scheme that a human reads in minutes. Ask each provider to name the test cases they run by hand, how they test every application role, and how they will disclose their methods in the report, because the ASVS asks certifying organizations to do exactly that.

Combine the two when you need broad coverage and a defensible penetration test on the same timeline. Run automated scanning on a schedule to catch drift, then put manual exploitation on the scope that changed or carries the most business risk. Under the CIS Controls, penetration tests are expected at least annually and detections should be checked against the techniques testers used, so the manual findings should feed your monitoring work as well. The main cost driver is tester time on the manual portion, so ask each quote to separate automated scanning effort from hours spent exploiting, and to state what retesting after remediation is included.

How manual and automated penetration testing compare on goals, method and evidence

This table compares manual, tester-driven penetration testing with automated and AI-assisted testing on goals, method, coverage and evidence. Read each row as a question to ask a provider, and note that the strongest contrast sits in business logic, access control and what the report can prove.

Factor	Manual penetration testing	Automated or AI-assisted testing
Primary goal	Validate whether a weakness can be chained into real impact on the target, including what an attacker could reach after the first foothold	Identify known weakness patterns quickly across many hosts or endpoints, creating a broad first pass for the tester to build on
Speed and repeatability	Slower and dependent on tester skill, but findings depend on judgment that a tester can explain and reproduce	PCI SSC notes automated tests run much faster than a human and repeat consistently, which suits regular runs between manual tests
Business logic and access control	Can test workflows, role boundaries and multi-step abuse. OWASP ASVS 5.0.0 notes automated DAST and SAST cannot fully verify many business logic and access control requirements	Limited for business logic and access control. Confirm with the provider which of these checks the tool actually performs before relying on it
Hidden or unusual functionality	Testers find undocumented parameters, custom authentication schemes and admin paths through review and manipulation. OWASP WSTG v4.2 gives such cases as examples automated black-box testing cannot reach	Effective on exposed, known patterns. Hidden parameters or home-made authentication schemes are often missed unless paired with code review
False positive and false negative handling	Tester confirms or rejects each finding by exploiting or demonstrating it, so the report separates verified issues from noise	Produces candidate findings that need triage. Ask who validates results and how unverified items are labeled in the report
Skill and people required	Requires experienced testers whose qualifications you should verify, such as CREST or other recognized credentials, and clear rules of engagement	Requires staff to configure, tune and interpret the tool, plus someone accountable for exclusions and scan credentials
What the report contains	Narrative of attack paths, proof of exploitation, scope covered, methods used and retest guidance. OWASP ASVS 5.0.0 expects certifying organizations to disclose testing methods in the report	Tool output such as findings lists and severity ratings. Usually insufficient alone as penetration test evidence unless the methods and manual validation are documented
Retest approach	Tester re-exercises each closed finding to confirm the fix works, which is the evidence assessors usually ask for	Rescan after remediation confirms a signature is gone, but may not prove that an exploit path has been closed
Fit with PCI DSS 11.4 and CIS Controls 18	PCI SSC describes penetration testing as essentially manual and says an automated tool alone does not satisfy the requirement. CIS Safeguard 18.4 asks that detections be validated against the techniques testers used	Supports the requirement as a first step in a network-layer test or as interim scanning, but does not replace the manual test itself. Confirm the scope with your assessor before relying on it

How a manual test and an automated scan actually differ in method

The core difference is what the work is for. NIST SP 800-115 separates review techniques, which are passive and carry minimal risk to systems, from target vulnerability validation techniques such as penetration testing, which use real exploits against production systems and data and carry high risk. An automated scan mostly belongs to the first group and identifies candidate weaknesses. A manual penetration test belongs to the second group and confirms which weaknesses can actually be used. When you write the statement of work, ask the provider which of its activities are scans, which are validation and who performs each one, because a report that mixes the two without saying so will overstate what was proven.

Mapping the PTES phases to effort makes the split concrete. Intelligence gathering and vulnerability analysis are where tools do most of the volume work, such as enumerating hosts, services and versions and matching them to known issues. Exploitation, post exploitation and much of threat modeling depend on a tester deciding what to try next, chaining weaknesses and judging whether access matters to your business. Reporting still needs a person to explain impact and remediation. Ask for a breakdown of hours or activities by phase so you can see where the tester's judgment was applied and where tool output was simply passed through.

PCI SSC describes automated tests as much faster than a human and as the logical first step in a network-layer test, but it says running an automated tool alone does not satisfy the penetration testing requirement. A useful illustration (hypothetical, for planning only): an application exposes an administrative function behind a hidden 30-character parameter. A scanner sends standard requests and sees nothing unusual, while a tester who reads the application's behavior, or its code, can find the parameter and test what it unlocks. OWASP WSTG v4.2 uses comparable examples to argue that automated black-box testing will never be fully effective, so confirm with any provider how their tester covers logic and access paths that a scanner cannot reach.

What manual testing finds that automation misses, and what it still misses

Automated tools are good at breadth and repetition, but several classes of weakness depend on understanding how an application is meant to behave. The cards below cover where manual work earns its cost, and where even a skilled tester runs out of coverage.

Business logic flaws

Logic flaws appear when every individual request is valid but the sequence or rules are not, such as applying a discount twice, skipping a payment step, or changing an order after approval. OWASP ASVS 5.0.0 states that automated DAST and SAST tools cannot fully verify many requirements, especially those on business logic. Ask the provider to show test cases built from your workflows, not only tool output.

Broken access control across roles

A scanner can request a URL, but it does not know that a customer should never see another customer's invoice. Testers need accounts for every role and must compare what each one can read and change across tenants and object IDs. Ask for a role matrix in the scope and confirm which roles were provisioned for testing.

Custom authentication schemes

Home-made login, token or single sign-on logic is often easy to read in code review but unreachable for a black-box scanner. The WSTG v4.2 gives the example of a homemade single sign-on hash scheme that a tester can spot by understanding how it works. Ask whether the test includes review of the authentication design, not only a password and session check.

Chained exploitation

Several low-severity findings can combine into a serious compromise, for example an information leak that reveals an internal path, which a second weakness then exploits. Automated tools report each finding in isolation and rarely connect them. Ask the provider to describe how findings were chained, and which attack paths were attempted and which were ruled out.

Hidden parameters and backdoors

The WSTG v4.2 notes that a hidden 30-character parameter can open an administrative backdoor that a black-box scanner will not guess. Discovery depends on reading client code, documentation and error messages, then trying the inputs by hand. Ask whether the scope allows source access or gray-box information to find these paths.

Coverage limits of manual work

Manual testing depends on time and tester skill, so depth varies across a large application. The WSTG v4.2 states that automated black-box testing is not and will never be completely effective, which means manual work is also bounded. Ask for a coverage list of tested functions and untested areas, and confirm the time allocated per application.

Which standards accept automated testing as a penetration test

Does PCI SSC accept an automated scan as a penetration test?

No. PCI SSC's Penetration Testing Guidance describes penetration testing as essentially a manual process and states that simply running an automated tool does not satisfy the penetration testing requirement. The guidance also says automated tests can be performed much faster than a human and are the logical first step in a network-layer test, so automation has a place in the workflow. Treat it as a starting point, not the finished test. Ask your QSA to confirm in writing which parts of the test they accept as automated, how manual exploitation was performed and documented, and how the report shows the tester's own analysis. The guidance is an information supplement, so the final call on how the requirement applies to your environment rests with your assessor.

Does OWASP ASVS 5.0.0 accept automated tools for verification?

ASVS 5.0.0 does not accept a tool run on its own as sufficient. It says certifying organizations may choose their testing methods but should disclose them in the report, and that merely running an automated tool without thorough testing is insufficient. It also states that automated DAST and SAST tools cannot fully verify many requirements, especially those on business logic and access control. In practice, a report that names a scanner as the only method will draw questions from assessors. Disclose each method, the requirements it covered, and where manual verification was performed. Ask your assessor which verification level applies, which requirements they expect to see verified by hand, and whether their report template asks for the testing methods.

Do CIS Controls 18.2, 18.4 and 18.5 accept automated tests?

The supplied facts do not show that an automated-only test meets these safeguards. Safeguards 18.2 and 18.5 ask for external and internal penetration tests at least annually, which is a test requirement and not a tool requirement. Safeguard 18.4 asks that security measures be validated after each test so that detections cover the techniques the testers used. That last point matters for automation. If testers only ran a scanner, the detections you validate cover scanner behavior, not the attack paths a skilled tester would take. Confirm with your assessor which tests they accept as penetration tests, how the annual frequency is evidenced, and whether post-test validation includes the manual techniques used in the engagement.

What does CREST's AI-Enabled Penetration Testing accreditation prove?

The supplied facts establish that CREST announced in September 2026 its first ten companies accredited for AI-Enabled Penetration Testing, and that CREST accredits member companies and certifies individuals in security testing disciplines. The supplied material does not set out what the accreditation requires. Do not assume it certifies fully automated testing, and do not assume an AI-assisted report removes the need for a human tester. Ask CREST and the provider for the accreditation criteria, which tests the accreditation covers, whether human review and exploitation are part of the service, and how the scope and method are recorded for each engagement. If a scheme you rely on names CREST, such as the Bank of England's CBEST arrangements, ask whether that scheme recognizes this specific accreditation.

Does DORA accept automated vulnerability scans as penetration testing?

DORA lists these as separate items. Article 25(1) names vulnerability assessments and scans, open source analyses, source code reviews where feasible, scenario-based tests and penetration testing as elements of the testing programme. Article 26 requires threat-led penetration testing at least every three years for entities that their competent authority identifies. Automated scans count toward the programme, but the supplied facts do not show that they satisfy the penetration testing element or the threat-led requirement. Confirm with your competent authority whether your entity is in scope for threat-led testing, which tests it expects from you, and what it requires of the testing provider in terms of scope, independence and method.

Can an automated tool meet a TIBER-EU threat-led red team test?

No. TIBER-EU is a threat intelligence-based red team test run in preparation, testing and closure phases, with at least 12 weeks of active testing and mandatory purple teaming. Scenarios are built from threat intelligence, and the work depends on tester-driven emulation of an adversary, which a scanner does not provide. The ECB states these tests produce no pass or fail result, so the output is a report of what the test found and how defenses responded, not a score. Ask the relevant authority or your TIBER-EU cyber team which providers they accept, how threat intelligence and red team providers are selected, and how the purple teaming phase and its evidence are documented.

Can I use automated testing alone to meet these standards?

Based on the supplied material, no. Automation adds coverage and speed, but each framework expects a penetration test with human judgment on top. NIST SP 800-115 suggests regularly scheduled network and vulnerability scanning interspersed with periodic penetration testing, and notes that organizations often combine techniques. A practical pattern is frequent automated scanning for known weaknesses, plus scheduled manual tests that cover business logic, access control and attack chains. Cost follows the split. Scanning tools carry recurring licensing or hosting costs, while manual testing is a scoped engagement priced by the work it includes. Ask any provider to quote the automated and manual work separately, with the methods, tools and deliverables listed for each.

Choosing the primary approach by situation

When a payment environment falls under PCI DSS, requirement 11.4 asks for penetration testing evidence, and the PCI SSC guidance describes penetration testing as essentially a manual process. Manual testing leads here. Automated tools still help, because the guidance calls them much faster than a human and the logical first step in a network-layer test, but running a tool alone does not satisfy the requirement. The condition that changes the answer is the assessor's expectation, so the scope must be agreed before testing starts.

Manual exploitation leads: Testers chain findings, validate exploitability and document the path toward cardholder data, which is the evidence an assessor reads.
Automation maps the estate first: Automated scans identify hosts, services and exposed components so manual effort goes where it matters most.
Method disclosure: Ask the provider to state which methods and tools were used, and to show the manual work separately in the report.
Segmentation verification: Manual testers confirm that out-of-scope networks cannot reach the cardholder data environment, and they retest fixes before closure.

Confirm with your qualified security assessor which test scope, methods and evidence format they expect before you sign a statement of work.

What drives the cost and effort of each approach

Manual and automated penetration testing cost different things, and the bill depends on who is paid for which hours and how often the work repeats. The cards below cover the main drivers, what is one-time and what recurs, and what to ask for in a written quote before any work starts.

Tester hours for manual exploitation

Most of the fee for a manual test is the time a qualified tester spends chaining weaknesses, validating business logic and proving impact. PCI SSC describes penetration testing as essentially a manual process, so this is the component that automation cannot remove. Ask the provider how many hours go to reconnaissance, exploitation and reporting, and who on the team holds the skills for each phase.

Scope size and number of roles

Cost rises with the number of applications, APIs, hosts and user roles in scope. OWASP WSTG v4.2 notes that automated black-box testing cannot find everything, so each additional role usually adds manual test cases. Ask the provider to list every application, environment and role it will test, and to state what happens if the scope grows during the engagement.

Production versus staging environment

Testing live systems usually needs more planning, change windows, monitoring and rollback arrangements than testing a copy, and those hours are billed or absorbed by someone. The environment also changes how representative results are. Confirm in the quote who provides the test environment, who pays for any staging build, and who coordinates production safety during testing.

Tool licensing and platform fees

Automated scanners and AI-assisted platforms often carry recurring licence or subscription fees, and some are charged per target or per seat. These fees do not replace the manual work PCI SSC and OWASP ASVS 5.0.0 expect for a penetration test. Ask whether the tool licence is included in the quote, who owns the licence, and whether results and data stay within your infrastructure after the engagement ends.

Retest effort after remediation

Retesting confirms that fixes work and is usually a smaller, recurring cost tied to each remediation cycle. CIS Controls Safeguard 18.4 asks that security measures be validated after each test, so detection and fixes need checking as well. Ask whether retests are included, how many rounds are covered, and whether automated re-scans are separated from manual confirmation of each finding.

Report depth and evidence

A short findings list costs less to produce than a report with reproduction steps, risk ratings, remediation guidance and evidence an auditor can follow. Automated output needs human triage before it is useful, and that triage time is part of the effort. Ask the provider what the report includes, whether it maps findings to the methods used, and what format auditors or internal teams will receive.

What you receive from a manual test versus an automated scan

This table compares what a tester-led penetration test delivers against what an automated scan produces, row by row. Read it as a checklist for the report you will actually receive, and ask each provider for a sample before you sign.

Deliverable	Manual penetration test	Automated scan output
Scoping and rules of engagement	Written scope, testing windows, permitted techniques and escalation contacts, following the rules of engagement approach in NIST SP 800-115 Appendix B	Usually a target list and scan profile; rules of engagement are often limited to credentials and IP ranges
Methodology disclosure	Named methodology and the techniques used, which assessors and PCI SSC expect the provider to state in the report	Scanner name, version and plugin or rule set, rarely the test logic behind each check
Validated findings	Each finding is exploited or confirmed by a tester, with the conditions under which it works	Findings are signatures or heuristics; many are unconfirmed and need triage
Reproduction steps	Step-by-step proof of concept that your team can rerun to confirm the issue	Generic description and request or response snippets, often without a working exploit path
Raw automated output	Usually attached as an appendix for transparency, not as the primary result	Core deliverable, typically an export in a machine-readable format for import into a ticketing or vulnerability tool
Risk ratings	Severity adjusted for the exploit path and business context, with rationale stated	Base severity from the scanner, often without environmental context unless you configure it
Business logic and access control findings	Tester reviews workflows, roles and authorization checks, areas OWASP ASVS says automated tools cannot fully verify	Limited coverage; findings in these areas usually need a human to confirm
Remediation guidance	Fix direction tied to the root cause and the affected component, with notes on verification	Vendor or generic fix text, sometimes a patch reference, with a high false positive rate to review
Retest report	Confirmation that each fixed finding no longer reproduces, usually a separate or addendum report; confirm whether retest is in scope	Rescan results; confirm whether a rescan proves the fix or only that a signature no longer fires
Evidence usable by an assessor	Scope, methodology, dated results, tester qualifications and retest status, which assessors can map to their requirements	Scan logs and reports; PCI SSC guidance says an automated tool alone does not satisfy the penetration testing requirement

Questions buyers ask before choosing manual or automated penetration testing

Do automated tools alone satisfy PCI DSS penetration testing?

No. PCI SSC's Penetration Testing Guidance v1.1 (September 2017) describes penetration testing as essentially a manual process and says that simply running an automated tool does not satisfy the penetration testing requirement. Automated scanning still has a place. The guidance says automated tests run much faster than a human and are the logical first step in a network-layer test. The tester then has to validate findings, chain weaknesses and exploit what is reachable. Confirm the exact requirement you are testing against with your assessor or acquirer before you agree on scope, because the test method has to be defensible in your Report on Compliance.

How much effort does a manual penetration test need from my team?

Manual testing needs more client time than a scan, mostly at the start and the end. Before the test, your team provides scope, network and application details, rules of engagement, and test accounts for each application role that should be tested. During the test, someone should be available to answer questions and approve any activity that could affect availability. After the test, your engineers retest fixes and your security team reviews the report. Ask the provider to list every client input in the written scope so you can staff it before work starts.

Can my team run automated security testing in-house instead?

Your team can run automated scanners and DAST or SAST tools in-house, and NIST SP 800-115 suggests regularly scheduled network and vulnerability scanning interspersed with periodic penetration testing. The limits are in the tools. OWASP ASVS 5.0.0 says automated DAST and SAST tools cannot fully verify many requirements, especially those on business logic and access control. Running an automated tool without thorough testing is insufficient for ASVS-based verification. In-house automation works well for frequent coverage. Independent manual testing still needs an external or independent tester where a standard or regulator requires independence, so confirm that requirement for your own program.

What drives the cost of manual versus automated testing?

Automated testing costs are mostly tooling, licenses and the staff who run and triage the output, and they recur as you scan more often. Manual testing is priced on the tester hours needed to cover the scope, and it is usually a one-time engagement per cycle. The main cost drivers are the number of applications and roles, the number of hosts and network segments, whether the work is internal or external, how much authenticated access is provided, and how much retesting is needed. DeployOpen does not publish a price list. Ask every provider for a written scope with deliverables, the tester hours by layer, and any retest rounds included, so quotes can be compared on the same basis.

How long does a manual penetration test take, and what affects timing?

Duration depends on scope, not on a fixed number of days. The factors that move timing are the number of applications and user roles, how many APIs and integrations exist, how complete the documentation is, how quickly test accounts and environments are provisioned, and how many findings need validation or retesting. Automated scans can run quickly, but triage and manual validation of their output still take tester time. Ask the provider for a timeline that is written into the scope, and confirm which dependencies on your side could delay the start, such as access approvals or change windows.

How should I combine manual and automated testing in one program?

Use automation for breadth and frequency, and manual testing for depth where logic and access decisions matter. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk. OWASP WSTG v4.2 says no single technique is enough and recommends a mix that puts more weight on early development phases. A practical pattern is scheduled scanning throughout the year, SAST and DAST in the build pipeline, and manual penetration testing that covers authenticated roles, business logic and the findings that automation cannot validate. CIS Controls v8.1 Safeguards 18.2 and 18.5 ask for external and internal penetration tests at least annually, and Safeguard 18.4 asks for detections to be validated after each test.

What does CREST AI-enabled penetration testing accreditation mean for choosing a provider?

CREST announced in September 2026 its first ten companies accredited for AI-Enabled Penetration Testing. For a buyer, the accreditation tells you that a named company has met CREST's requirements for that service, so you can check the public list and ask the provider to show its accreditation scope. It does not replace scope review. Ask which parts of the test use AI-assisted tooling, who reviews and validates AI-generated findings, how data sent to third-party AI services is handled, and whether the tester is accountable for the final report. Confirm current accreditation status directly with CREST before you rely on it.

How to combine automation and manual testing without gaps

Automation and manual testing answer different questions, so a gap-free program assigns each target a layer on purpose. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk, and it suggests regularly scheduled network and vulnerability scanning interspersed with periodic penetration testing. In practice, scanners and dynamic or static analysis tools cover the broad, repeatable sweep: known vulnerable versions, misconfigurations, missing patches and common injection patterns across every host and build. Manual testers take the parts that depend on context: business logic, access control across each application role, chained weaknesses and anything that needs a valid session. The OWASP ASVS 5.0.0 guidance says automated DAST and SAST tools cannot fully verify many requirements, especially business logic and access control, so those items belong on the manual scope list.

A practical buyer's checklist starts with the application inventory. For each internet-facing application, API and internal system, confirm which roles exist, which workflows move money or personal data, and which functions a scanner cannot reach without credentials. Put those on the manual scope, and ask the provider to name the test accounts it will use and how they were provisioned. Automate the rest on a schedule, and ask the scanning vendor to show which checks it runs authenticated and which it runs anonymously. Then validate coverage. CIS Controls Safeguard 18.4 asks that security measures be validated after each test so detections cover the techniques the testers used, so map each manual finding to the detection rules or alerts that should have fired. Ask the provider to log its techniques against MITRE ATT&CK, and confirm that your monitoring team can see the same activity.

Watch for two gaps that a clean scan report hides. The first is a manual test that was scoped as a scan with a human reviewing the output. PCI SSC guidance says running an automated tool alone does not satisfy the penetration testing requirement, so the quote and statement of work should state that exploitation and business logic testing are performed manually and name the testers' roles. The second is a scanner that runs on a schedule but never sees the newest application releases. Tie each release to a rescan and a manual review of changed functions. Ask for a retest of findings and a written statement of which items were not tested and why, so the next test starts from known limits rather than assumptions.

How DeployOpen works on this

DeployOpen is a Chainscore Labs company that plans, runs and documents security testing inside the client's own infrastructure, including penetration testing. We scope manual and automated work together in writing before any testing starts, so each approach is matched to the systems it can actually validate.

How we scope

Written scope states which systems, roles and layers get manual exploitation and which get automated scanning.
Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work begins.
Scope records how each application role will be tested, since logic and access control checks need human judgment.

How we build

Automated scanning runs first as a baseline, then testers validate findings and attempt manual exploitation beyond scanner output.
Test accounts for each application role are provisioned under agreed rules, with access limits documented before testing starts.
Testers record the methods used, so the scope can show which techniques were manual and which were tool-driven.

How we hand over

Reports separate scanner output from manually validated findings, with reproduction steps and affected components for each.
Deliverables document the methodology, tools, versions and limitations, so an assessor can check how the test was performed.
Handover includes retest guidance and a list of detections to confirm, so monitoring can be checked against the techniques used.

How we support

Ongoing support, upgrades and continuous testing are available as retainers after the initial engagement.
Retainers can schedule recurring automated scans between periodic manual tests, with the manual cadence agreed in writing.
Confirm with DeployOpen during scoping which retest windows, reporting formats and support hours a retainer includes.