Safeguard 18.2 External penetration tests: evidence and audit

What Safeguard 18.2 external penetration tests require

If your enterprise is in Implementation Group 2 or 3 and a customer, auditor or regulator asks you to show annual external testing, the short answer is this: Safeguard 18.2 expects periodic external penetration tests, at least once a year, run by a qualified party, with reconnaissance aimed at finding information an outsider could exploit. The tests may be clear box, where the tester receives documentation and internal context, or opaque box, where the tester starts with little more than the target. The Safeguard sits in Implementation Group 2, so IG1 enterprises are not measured against it, but IG2 and IG3 enterprises are.

The risk it addresses is the gap between what your team believes is exposed on the internet and what an attacker can actually reach. Externally facing hosts, forgotten subdomains, VPN portals, cloud storage endpoints and third-party integrations change faster than most teams track them. The hard parts are usually operational: booking a tester before the twelve-month window lapses, keeping the scope aligned with the assets that really face the internet, and making sure findings land in a tracked remediation queue rather than a PDF. The CIS Controls Assessment Specification measures this Safeguard as failing once more than twelve months pass since the last external test, so a missed booking is a visible gap, not a paperwork delay.

Before you commission a test, confirm three things with your own records: the date of the last external test, the list of internet-facing assets in your current inventory, and whether the tester is qualified under the criteria in your program. The Safeguard 18.1 program should set the scope, frequency and remediation routing that these checks depend on. Sign-off on a test report is an independent matter: the organization owns its controls, an implementation partner can prepare systems and evidence, and the tester's report is evidence of testing, not a certification of compliance.

Which systems, teams and providers fall inside external testing scope

Safeguard 18.2 applies from Implementation Group 2 and is governed by the scope set in the penetration testing program under Safeguard 18.1. The cards below cover the decisions that most change the amount of work: what is reachable from the internet, which cloud accounts and domains count, which hosted services belong to a third party, and how much knowledge the testers receive.

Internet-facing assets

Start with what an outside attacker can actually reach: public web applications, VPN gateways, remote access portals, mail gateways and any service that answers on a public address. Build this list from scanning and network records, not only from the asset inventory, because forgotten or temporary hosts often sit outside it. Ask the testers to confirm which addresses are in scope and which are excluded, and record the reason for each exclusion so an assessor can follow the decision.

Cloud accounts and subscriptions

Public cloud accounts often expose services that never appear on a data center diagram, such as storage endpoints, load balancers and serverless functions. Confirm which accounts, subscriptions and regions the tests may touch, and whether the cloud provider's testing terms apply. Name a cloud owner who can approve the test window and can disable a service if the test causes an outage, since this decision changes both the test plan and the rules of engagement.

Domains and IP ranges

Scope is defined by the exact domains, subdomains and address blocks the organization owns or controls. Reconcile registrar records, DNS zones and address allocations against the list before testing, because a subdomain pointing to a retired host is a common route into an environment. Write down any wildcard domains and any ranges shared with another company, since those need separate written permission before anyone probes them.

Third-party hosted services

Software-as-a-service platforms, managed hosting and outsourced portals may sit under your brand but run on infrastructure you do not operate. Identify which of these the provider allows you to test, which require the provider's own approval, and which are out of scope entirely. Where the provider prohibits testing, document that limit and the compensating checks you rely on, such as contractual security reports or assessments the provider shares with you.

Clear box or opaque box approach

Safeguard 18.2 allows external tests to be clear box, where testers receive architecture details, configurations or credentials, or opaque box, where they start with no internal knowledge. Choose the approach in the 18.1 program and state it in the test scope. Opaque box tests show what a real outsider could learn, while clear box tests cover more of the attack surface per day. Confirm with the testing party which approach they will use and what information you must provide.

Teams that approve and respond

External testing touches several teams: network, application owners, cloud operations, the security operations center and sometimes legal or the provider managers. Confirm who signs off the scope, who can stop a test, who receives findings and who must be warned so that alerts from the test are not treated as a live incident. Gaps here are a frequent cause of delayed or disputed tests, so record the contacts in the program documentation.

How the CIS assessment specification measures 18.2

The CIS Controls Assessment Specification measures Safeguard 18.2 at zero and fails it when more than twelve months have passed since the last external penetration test. That single rule sets the clock for the whole safeguard. An assessor does not grade the quality of a test in this measure. They check the date of the most recent external test against the date of the assessment, and a test that falls outside the twelve-month window fails regardless of what it found. The first decision for the enterprise is therefore a calendar decision: record the completion date of every external test, set a reminder well before the twelve-month mark, and confirm that the engagement with the testing party can be scheduled ahead of that date.

The specification also depends on Safeguard 18.1, which it lists as a prerequisite for measuring 18.2. In practice an assessor will first look for the penetration testing program that 18.1 requires: a documented scope, frequency, limitations, contacts and remediation routing. Without that program, a dated external test is harder to defend as part of a managed process. Expect the assessor to ask for the program document, the approval record, and the list of external tests with their dates, scopes and the party that performed each one. Confirm with the testing provider that its report states the testing type (clear box or opaque box), the reconnaissance performed, and the qualifications of the testers, since those are the details a reviewer will look for when checking that the test was run by a qualified party.

The specification covers what to measure, not how. It tells an assessor which inputs to collect and which result to record, but it does not prescribe a testing methodology, tool, report template or reconnaissance technique, so the enterprise must define those in its own program and be able to explain them. Ask the testing party to document how it performed reconnaissance for exploitable information across the enterprise and its environment, and keep that documentation with the report. Because the measure cannot tell an assessor whether the test was thorough, it is worth confirming how the scope was set and who approved it, since a reviewer may ask how the enterprise decided what the external test would and would not cover.

How an assessor tests Safeguard 18.2

This table compares the points an assessor checks for Safeguard 18.2, what evidence each point needs and where submissions usually fall short. Read each row as a question the assessor will ask, then confirm the answer with the evidence in the right column before the assessment begins.

Assessment point	What the assessor checks	Evidence usually requested and common gaps
Test date against twelve-month window	The most recent external penetration test is no more than twelve months old when measured.	Signed test report with start and end dates. Gap: a test scheduled for a date that slipped, or a report that is undated or covers only a partial scope.
Tester qualification	The test was performed by a qualified party with specialized skills and experience, as the Safeguard requires.	Tester credentials, engagement letter and statement of experience. Gap: an internal team without documented specialist skills, or no evidence of how qualification was judged.
Scope and reconnaissance coverage	The external scope covers enterprise and environmental reconnaissance aimed at finding exploitable information.	Scope document listing public IP ranges, domains, cloud endpoints and exposed services, plus reconnaissance output. Gap: scope limited to a few hosts with no record of how the perimeter was identified.
Clear or opaque box designation	The report states whether the test was clear box (with provided information) or opaque box (without it).	Rules of engagement that name the test type and the information given to testers. Gap: the designation is missing from the report.
Findings and remediation link to 18.3	Findings are recorded and routed into the documented remediation process under Safeguard 18.3.	Findings register with owners, severity and remediation status, linked to tickets. Gap: findings that sit in the report without tracked remediation or retest records.

How to run an external penetration test that passes

This sequence moves from an approved penetration testing program to a dated, evidenced external test that the assessor can trace from scope through remediation to the next scheduled test. Each step produces a record that shows Safeguard 18.2 is met.

Confirm the program and scope
Start by checking that the penetration testing program under Safeguard 18.1 is written and approved, since the assessment specification treats 18.1 as a prerequisite for measuring 18.2. Document the external assets in scope (public IP ranges, domains, cloud-hosted endpoints and internet-facing applications), the test frequency and the named owner. The step closes when the scope statement is signed off by the accountable manager and filed with the program records.
Select a qualified testing party
Safeguard 18.2 requires a qualified party with specialized skills and experience, so the organization must be able to show why the tester is qualified. Ask providers for their named testers, relevant certifications and experience with external assessments, and confirm whether they are independent of the teams that run the systems under test. Whether the tester is an internal team or an outside firm, the output is a written selection record that an assessor can review.
Agree rules of engagement and test approach
Document the rules of engagement before any testing starts: permitted techniques, testing windows, out-of-scope systems, emergency contacts, data handling and stop conditions. Decide whether the test is clear box, with credentials and design documents provided, or opaque box, with only the public starting point given. The output is a signed rules of engagement document that states the chosen approach and why it fits the program's objectives.
Run reconnaissance and testing
Reconnaissance is part of the requirement: the tester must look for enterprise and environmental information that could be exploited, such as exposed services, leaked credentials, misconfigured DNS records and version details. The tester then attempts to exploit weaknesses within the agreed rules. The organization's monitoring team should log test activity so the results can later be compared with alerts. The step closes when the tester confirms testing is complete and the activity log is preserved.
Receive and review the test report
The tester delivers a written report that lists each finding, the evidence supporting it, the affected assets, the exploit path and a severity rating with recommended fixes. The organization reviews the report for accuracy and asks the tester to clarify any finding that cannot be reproduced. The output is a final report accepted by the program owner, with false positives recorded and removed from the remediation queue.
Route findings to remediation
Enter each validated finding into the documented vulnerability remediation process, which assigns an owner, a target date and a status for each item. Findings that the team cannot fix must go through a formal risk acceptance with an approver. Safeguard 18.3 covers the remediation timelines in detail. Teams often track these items in a findings platform so that duplicates and retests are easy to show, and the step closes when every finding has a tracked outcome.
Schedule the next test within twelve months
The assessment specification measures Safeguard 18.2 as failed when more than twelve months pass since the last external test, so the next test must be booked before the current cycle ends. Record the date of the completed test, set a calendar reminder well before the deadline and update the program schedule. The step closes when the next test date is recorded in the program and the evidence file is updated for the assessor.

Evidence that shows an external test was performed

An assessor testing Safeguard 18.2 wants a dated, traceable record that a qualified party tested the external perimeter, that the findings were handled, and that the next test is scheduled inside the twelve-month window. The cards below cover the artifacts that prove each link in that chain, and the habits that keep them current between audits.

Signed scope and rules of engagement

The signed scope lists the external IP ranges, domains, cloud accounts and applications in test, along with exclusions, testing windows, permitted techniques and the emergency contact. It is the document that proves the test was authorized and bounded. Check that the signatories are from the organization and the tester, that the date precedes the test window, and that the scope matches the asset inventory used for the program under Safeguard 18.1. A scope that omits internet-facing assets the organization later discovered is a finding in itself.

Tester qualification statement

Safeguard 18.2 requires a qualified party with specialized skills and experience, so the file should hold a written statement of who performed the work, their relevant credentials or experience, and whether they are internal staff or an external firm. Keep the statement tied to the named testers on each engagement, not a general company brochure. Record any independence arrangements, such as testers who did not build the systems they tested. Ask the provider to confirm these details in writing before work starts.

Dated test report

The report states the test dates, the methodology, whether the test ran as clear box or opaque box, the in-scope targets and the findings with severity and evidence. An assessor checks that the report date falls within twelve months of the assessment date and that the scope in the report matches the signed scope. Reports with no dates, no scope or only a generic risk score are weak evidence. Store the final version and the tester's sign-off together.

Reconnaissance records

Safeguard 18.2 includes reconnaissance for exploitable information across the enterprise and its environment. Keep the tester's recon outputs: discovered hosts, exposed services, certificate and DNS records, leaked credentials or documents found, and the sources used. These records show that reconnaissance went beyond the asset list and show what an attacker could have learned. Agree in the rules of engagement how long the tester may retain this material and how it is destroyed afterward.

Remediation tickets

Each finding should link to a ticket in the vulnerability remediation process with an owner, a priority, a target date, the fix applied and the closure approver. Assessors sample tickets to confirm that findings were tracked to closure rather than filed away. Tickets that record risk acceptance need a named approver and an expiry date. Where a finding was not fixed, the ticket should show the compensating control. Vulnerability tools such as DefectDojo can hold this linkage if the team configures it.

Retest results and the twelve-month calendar

A retest report shows which findings were verified closed, with the date and method of verification. Pair it with a calendar or schedule that records the last external test, the next due date and the owner responsible for booking it. Because the assessment specification fails Safeguard 18.2 once more than twelve months pass since the last test, the calendar is the control that prevents a gap. Set reminders well before the due date and record any reschedule with a reason.

How open-source tools support external test evidence

Safeguard 18.2 is met by a qualified party's external penetration test, and the organization needs a place to keep that party's findings alongside its own remediation records. DefectDojo (BSD-3-Clause) serves as that register. It imports and reimports findings from more than 500 security tool report formats through its UI or API, including in the open source edition, and it accepts a generic findings import for reports that do not match a supported format. Its deduplication uses per-tool hashcode fields, so a retest that reproduces an earlier finding updates the existing record rather than creating a duplicate.

Import path: Load the tester's report or a machine-readable export so each finding keeps its date, scope and severity.
Reimport after retest: Reimport results so the history of a finding shows when it was reproduced and when it was closed.
Generic import: Use the generic findings format for reports that arrive as spreadsheets or custom documents.
Ownership fields: Assign each finding to a remediation owner, which supports the routing expected under Safeguard 18.1.
Audit trail: Export the register as evidence that the test occurred within the twelve-month window.

Confirm with your tester which report formats they produce and whether the open source edition covers the import method you need.

What an external penetration test program costs to meet

This table lists the cost components of meeting Safeguard 18.2, who is paid for each one, whether it is one-time or recurring, and what drives it up or down. No figures are given because none were supplied for this safeguard, so use the table to build a quote request and compare written scopes. The main takeaway is that the tester fee is only one line, and internal scoping, remediation and evidence upkeep often decide the total effort.

Cost component	Who is paid or who spends effort	One-time or recurring	What drives it
Tester fees	Qualified external penetration testing party	Recurring, at least annually	Number of external addresses and services, depth of testing, clear box or opaque box approach, and retest rounds
Scope and reconnaissance effort	Tester during engagement; internal owner for scope sign-off	Recurring with each test cycle	Size of the external footprint, cloud accounts and third-party hosted assets, and how current the asset inventory is
Open-source tooling	Internal team or implementation partner for setup; license spend removed for self-hosted tools	One-time setup, then recurring operations	Number of tools deployed, hosting size, and staff time to run scans and import results into a findings tracker
Internal engineering time for scoping	Internal security and infrastructure staff	Recurring with each test cycle	Rules of engagement, testing windows, exclusions, and contacts for escalation during testing
Remediation effort	Internal system owners and engineering teams	Recurring with each test cycle	Number and severity of findings, and whether fixes need change windows or vendor involvement
Evidence upkeep	Internal compliance or security lead; partner if preparing evidence	Recurring, usually with each test	Retention of reports, scope letters, tester qualifications, and retest records for the assessor
Retest and validation	Tester for retest; internal team for fix verification	Recurring, after remediation	Number of open findings, complexity of fixes, and whether the same tester retests

Who does what for external penetration testing

Your organization owns Safeguard 18.2. Its leadership sets the penetration testing program under Safeguard 18.1, decides the scope and frequency of external tests, selects the qualified party that performs them, and accepts the risk of any finding left open. Your security function also owns remediation routing, so findings reach the people who fix the affected systems under your vulnerability remediation process.

Program ownership: Approve the written program, including the external test cadence, which must be no less than annual, and name an accountable owner.
Scope decisions: Define the external assets in scope, the reconnaissance boundaries and whether tests are clear box or opaque box.
Tester selection: Choose a party with specialized skills and experience, and keep the engagement letter and rules of engagement on file.
Remediation and acceptance: Track each finding to closure or to a documented, approved risk acceptance.
Records: Keep the test reports, dates and sign-offs that an assessor will ask to see.

Before the assessment, confirm that someone internal can show the last external test date, its scope and what happened to every finding.

Where external testing programs usually fall short

Safeguard 18.2 is easy to state and easy to let drift. The gaps below are the ones assessors and internal reviewers tend to find first, and each one breaks the chain from scoping to a dated, qualified, documented test. Check them before the next test window opens, not after the report arrives.

Test dates slipping past twelve months

The Safeguard requires external penetration tests no less than annually. The CIS Controls Assessment Specification measures 18.2 as failing once more than twelve months have passed since the last external test, so a test that finishes a few weeks late is still a failed measure. Track the completion date of each test, not the date the engagement was booked, and set the next window while the current report is still open. Confirm the report close-out date is recorded in your evidence register.

Scope missing new cloud accounts

External scope is usually written once against a list of public addresses and domains. New cloud accounts, load balancers, regional endpoints and SaaS-hosted subdomains appear between tests and quietly fall outside the scope the tester was given. Compare the scope document against your current asset inventory and cloud account list before each engagement, and ask the tester to record any internet-facing asset they find that the scope does not cover. Ask who approves scope changes mid-engagement.

Reconnaissance not documented

Safeguard 18.2 includes reconnaissance for exploitable information across the enterprise and its environment. When a tester's passive and active discovery steps are not written down, you cannot tell whether the enterprise footprint was actually mapped or only the systems the team already knew about. Require a written reconnaissance section in the report that lists the sources used, the domains and address ranges discovered and the exposures that were judged out of bounds. Illustrative example: a report that names no certificate transparency or DNS sources has likely missed subdomains.

Findings not routed to remediation

A clean report that sits in a shared folder does not satisfy the program. External findings need an owner, a severity, a remediation target and a retest result, and they should enter the same vulnerability workflow used for scanner output. Map each finding to a ticket when the report is delivered and record the retest outcome against it. Open-source tools such as DefectDojo can hold imported findings and deduplicate them across tests, so the history stays in one place.

Testers without evident qualifications

The Safeguard requires that a qualified party with specialized skills and experience perform the test. Procurement often checks price and turnaround, while the qualifications behind the named testers go unexamined. Ask for the names of the people who will do the work, their relevant certifications or documented experience in external testing, and whether they are employees or subcontractors. Confirm the test team has no conflict of interest with the teams that administer the systems under test. Qualifications should be verified before the statement of work is signed.

Clear box and opaque box choices undocumented

Safeguard 18.2 allows clear box or opaque box testing, and the choice changes what the report can prove. An opaque box test shows what an outsider could reach, while a clear box test with documentation shows how deep an attacker could go once inside the perimeter. Record which approach was used for each target, why it was chosen and what information the tester was given. A report that does not state its approach leaves an assessor unable to judge whether the test met the intent of the Safeguard.

Questions buyers ask about Safeguard 18.2

Does Safeguard 18.2 apply to IG1 organizations?

No. Safeguard 18.2 sits in Implementation Group 2, so it applies to IG2 and IG3 enterprises but is not part of the IG1 essential cyber hygiene set. IG1 organizations can still run external tests voluntarily, and doing so is a sound way to learn where their exposure sits. If a customer or regulator asks for evidence against 18.2 specifically, first confirm which Implementation Group the organization has adopted, because that determines whether the Safeguard is in scope at all. The CIS Controls Navigator shows the mapping for each Safeguard.

How often must an external penetration test be run under Safeguard 18.2?

At least once every twelve months. The Safeguard calls for periodic external tests set by program requirements, with no interval longer than annually. The CIS Controls Assessment Specification measures 18.2 as failing if more than twelve months have passed since the last external test, so the date of the most recent test is the number an assessor checks first. Many organizations set a shorter cadence after major changes to internet-facing systems, but the annual minimum is the baseline the specification measures.

Can in-house staff perform the external penetration test?

Only if the work is done by a qualified party with specialized skills and experience. CIS requires that penetration testing be conducted by someone qualified, and it does not require an outside firm by name. An internal team can qualify if its members have demonstrable testing experience and the test is planned under the penetration testing program defined in Safeguard 18.1. Ask yourself whether an assessor would accept the team's credentials and independence. Organizations that lack this depth often bring in an outside tester for the external test and keep internal staff for remediation.

What does the work need from our team?

Your team needs to define scope, provide authorization, and supply the reconnaissance inputs the Safeguard expects. Before testing, you agree on the external address ranges, domains, cloud accounts and third-party hosted assets in scope, and whether the test is clear box (tester given documentation or credentials) or opaque box (tester starts with no inside knowledge). Your team must also confirm rules of engagement, emergency contacts and what happens if a critical finding appears mid-test. After the test, staff triage findings and route them into the remediation process. Plan for this time up front, because scope disputes are a common source of delay.

What drives the time and effort of an external test?

The main drivers are the size of the external attack surface, the test type, and how much remediation follows. A test covering a few public services takes far less effort than one covering many domains, cloud accounts, APIs and third-party-hosted systems. Clear box tests take more preparation to brief the tester, while opaque box tests spend more time on reconnaissance. Scope changes during testing, retesting after fixes, and the reporting depth you require all add effort. Get a quote from any provider that breaks effort down by asset group, test type and retest rounds, so you can compare offers on the same basis.

Does a vulnerability scanner satisfy Safeguard 18.2?

No. A scanner is useful for finding known weaknesses, but Safeguard 18.2 asks for a penetration test by a qualified party that includes reconnaissance for exploitable information. Automated tools support that work. Greenbone Community Edition (OpenVAS) schedules scans, stores results in PostgreSQL and manages users, and DefectDojo can import findings from more than 500 report formats and deduplicate them. Wazuh and Fleet can flag vulnerable software on monitored hosts. These produce evidence for Control 7 and inputs for testing, but a human-led test is what 18.2 describes.

How does an auditor check the date of our last external test?

The auditor looks for a dated record of a qualified external test within the last twelve months, and then checks that the record matches your stated scope. Expect to provide the engagement letter or scope, the test report with its start and end dates, the tester's qualifications, and evidence that findings were routed to remediation. Under the Assessment Specification, the measure is whether more than twelve months have passed since the last test, so the report date matters more than the date you intended to test. Keep these records together in one place before the assessment begins. Only the independent assessor or auditor reaches the conclusion.

How DeployOpen prepares external test evidence

DeployOpen plans, prepares and supports the external test program, but it does not issue reports, attestations or certificates. For Safeguard 18.2, the organization owns the decision to test, the scope, the qualified party it engages and the remediation that follows. DeployOpen's part is to agree the scope in writing before work starts, set out which systems, cloud accounts and internet-facing services are in and out of bounds, and record the test window, contacts and rules of engagement so the test can be shown to be periodic and no less than annual. Any written scope should state whether the test is clear box or opaque box and why, since that choice changes what the tester is given and what the evidence must show.

Inside the client's own infrastructure, DeployOpen can prepare the systems and tooling that produce the records an assessor will ask for. This typically includes the scanning and vulnerability management platforms named in this section, configured to keep dated results, the tester's findings and the remediation tickets linked to each engagement. Security testing itself can be delivered as part of the engagement, under the same written scope. Before the first test, confirm which party employs the testers, what their qualifications are, and how the test report will be stored and handed to the independent auditor or assessor, who decides whether the evidence is sufficient.

DeployOpen does not issue audit reports, attestations or certifications, and its work does not replace an assessment by an independent auditor or certification body. If you are asked to show Safeguard 18.2, ask any implementation partner to state in writing which evidence it prepares, which it only retains, and which must come from an independent party. The CIS Controls Assessment Specification is the reference to check that each evidence item maps to the measure it supports.

Safeguard 18.2 External penetration tests