Cloud configuration review or cloud penetration testing: which do you need first?
A configuration review checks whether your cloud account settings match a benchmark, such as the CIS Foundations Benchmarks. A penetration test checks whether an attacker can actually reach data or identities through those settings. If your team is regulated and has cloud accounts on a shortlist, start by asking which question your auditors and risk owners need answered first. A configuration review is the usual first step when you have no recent baseline, because it is faster, lower risk and covers every account in scope. A penetration test is the right next step once the baseline is clean and you need proof that identity paths, exposed workloads and misconfigurations chain together into real access.
Choose a configuration review first if you need a documented baseline for IAM roles, logging, encryption, network exposure and storage permissions across all accounts. NIST SP 800-115 classes such reviews as passive techniques with minimal risk to systems, and notes that automated checks are faster than manual ones, though some settings still need manual verification. Both methods need elevated privileges to read the selected security settings, so confirm with your provider who will hold that read access and how it will be logged. Choose a penetration test first if a regulator, board or customer asks whether an outsider or a compromised low-privilege user could reach regulated data.
Most regulated teams end up combining both. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk. In practice, run the review to close known gaps, then run a penetration test that tries to exploit what remains. Before any attack testing starts, read each provider's rules of engagement, since the permitted and prohibited activities differ, and agree the scope and notice terms in writing with your provider.
How the two approaches compare on scope, access, method and evidence
This table compares a cloud configuration review against a cloud penetration test on goal, scope, access, method, findings, limits, provider rules, deliverables, skills and cadence. Read each row as a question to put to a provider or internal team, since most mature programs run both.
| Dimension | Cloud configuration review | Cloud penetration test |
|---|---|---|
| Primary goal | Confirm account and service settings match a chosen benchmark, such as the CIS Foundations Benchmarks | Show whether an attacker can reach workloads, identities or data by chaining weaknesses |
| Scope | Accounts, subscriptions, projects, IAM policies, logging, encryption and network settings | Named applications, workloads, identities and endpoints the provider's rules allow to be tested |
| Access required | Read-only audit roles or equivalent, plus privileged access where selected settings can only be seen with elevated rights | Test accounts with defined permissions, often starting from a low-privilege or external position |
| Method | Automated benchmark checks, supplemented by manual review of settings the tools cannot read | Manual exploitation and tool-assisted testing within the agreed rules of engagement |
| Typical findings | Public storage, overly broad roles, disabled logging, unencrypted resources, unmanaged keys | Exploitable misconfigurations, credential exposure, privilege escalation paths, lateral movement across roles |
| What it misses | Whether a finding is exploitable in combination, and attacks on application logic | Full coverage of every setting, and a point-in-time view of drift after the test ends |
| Provider rules of engagement | Generally no notice needed for read-only review, but confirm provider policy and the account owner's internal approvals | Must follow provider policy, which may prohibit denial of service, certain DNS or subdomain actions and fuzzing |
| Deliverable | Control-by-control results against the chosen benchmark, with evidence of each setting checked | Narrative of attack paths, proof of impact, severity ratings and remediation sequence |
| Skills needed | Cloud platform administration, benchmark interpretation and evidence handling | Offensive cloud and identity skills, plus clear reporting for engineers and executives |
| Repeat cadence | Continuous or frequent, since settings drift with each change | Periodic, and after major architecture changes, as set by the program's risk appetite |
What each approach finds and what it misses
Configuration review and penetration testing answer different questions about the same cloud account. The cards below set out what each one surfaces, where each one falls short, and why the gaps usually line up, so the two can be scheduled and scoped together.
Over-permissive policies
A configuration review reads IAM policies, role trust relationships and resource policies and flags wildcard actions, broad principals and grants that exceed the job. Illustrative example: a workload role granted write access to every storage bucket in an account because one bucket needed it. The review sees the grant on paper. It does not show whether a reachable path turns that grant into something worse, so check which findings are reachable from an actual identity.
Attack chains across identities
A penetration test starts from a foothold, such as a leaked access key or a compromised workload, and tries to move between identities until it reaches a target. Illustrative example: a low-privilege role can assume a second role whose permissions were never reviewed alone. Each permission can look acceptable in isolation, and only the chain shows the exposure. Configuration review tends to miss these chains because it evaluates resources one at a time.
Public storage and network exposure
Configuration review checks whether buckets, snapshots, databases and security groups are reachable from the internet or from unexpected networks. This is fast to verify and easy to repeat. A penetration test shows whether an exposed resource holds data a tester can actually read and whether it leads somewhere else. Confirm with the provider which tests are permitted against storage and DNS, because some providers restrict takeover-style and flooding activity.
Logging and detection gaps
Configuration review checks whether audit logs are enabled, retained and sent to a location the workload cannot alter. It finds trails that are switched off or limited to one region. A penetration test shows whether an attack produces any alert at all. Ask the testing team to record each technique it used so the security team can confirm detections cover those techniques after the test.
Missing guardrails and drift
Configuration review finds the absence of guardrails such as service control policies, required encryption, tagging enforcement and approved-region limits. It also catches drift when a setting changes after the last review. Penetration testing rarely finds this kind of drift, because a tester uses the account as it stands on the test window. Schedule configuration checks on a recurring basis, not only before a test.
Settings off the tested path
A penetration test covers the paths the scope and the time window allow. A configuration setting on a service the tester never touched, or on a region outside scope, can stay unreviewed. Illustrative example: an unused data pipeline keeps a broad access key that no test path reaches. Configuration review is the method that covers the full inventory, so the scope note should state which accounts, regions and services the test does not reach.
How AWS, Microsoft and Google Cloud rules shape what testers may do
Each provider's testing policy sets the outer boundary of any cloud penetration test, and that boundary often decides which attack paths a tester can try. AWS prohibits DNS zone walking, denial of service, port, protocol or request flooding, and S3 bucket or subdomain takeover in customer testing. It also requires two weeks' notice for simulated events such as red team exercises, so a team planning an adversary simulation on AWS has to build that lead time into the schedule. Microsoft's cloud penetration testing rules of engagement prohibit DDoS attacks, network-intensive fuzzing, and any access to data or tenants the tester does not own. Microsoft also requires vulnerabilities found in Microsoft services to be reported to MSRC, so a tester who finds one has a reporting duty rather than a free choice of disclosure.
Google Cloud takes a lighter approach to notice. Its guidance states that customers do not need to notify Google before penetration testing their own Google Cloud infrastructure, but testers must follow the Acceptable Use Policy and Terms of Service and must affect only the customer's own projects. For a configuration review, these rules mostly matter when a check touches shared services or third-party resources. For a penetration test, the project boundary is the practical scope limit: a test that pivots into another team's project or a partner tenant falls outside the rules even if the attacker path looks valid.
Treat these summaries as a starting point, not as current policy. Provider rules change, and the text you rely on in a statement of work should be the version the provider publishes on the day you sign. Before scoping, ask your tester to confirm the current AWS, Microsoft and Google Cloud policies, the notice periods that apply to your planned tests, and the prohibited activity list for each service in scope. Record the confirmed versions in the rules of engagement so that a finding can be checked against the policy in force when the test ran.
How most cloud assessments combine a configuration review with a penetration test
The sequence turns a configuration review into a list of attack paths, tests the paths that matter, and then confirms that fixes and detections hold. Each stage produces an input for the next, so the final report shows what was misconfigured, what an attacker could actually reach, and what changed afterward.
Agree scope and rules of engagement
The client and the testing provider write down which cloud accounts, subscriptions or projects are in scope, which services and identities may be tested, which activities are excluded, and who receives alerts and stop requests. NIST SP 800-115 provides a rules of engagement template in Appendix B that teams commonly adapt. Before the work begins, the provider checks the target provider's penetration testing policy, since it sets which activities are permitted and whether notice is required. The step is done when the signed rules of engagement name the accounts, the testing windows, the prohibited tests and an emergency contact.
Baseline configuration and identity settings
The reviewer collects configuration and identity data from the in-scope accounts, using automated benchmark checks where they exist and manual checks for settings the tools cannot read. NIST SP 800-115 notes that automated configuration reviews are faster than manual ones, but some settings still need manual checks, and both methods require root or administrator privileges to view selected security settings. The reviewer should confirm with the client who will grant that read access and for how long. The step is done when every in-scope account has a recorded baseline and each failed check has an owner, a severity and a note on how it was verified.
Map failed checks into candidate attack paths
The tester reads the baseline and links individual findings into sequences an attacker could plausibly follow, such as an overly broad role that can be assumed from a workload, a storage bucket that is readable by a wider group, or a credential that reaches a privileged management API. This is analysis, not exploitation, and it is performed by the tester with the client's architects. The output is a short, ranked list of paths with the assumptions each one depends on. The step is done when each path names its entry point, the identity it needs, the asset it reaches and the check that raised it.
Test the raised paths within permitted limits
The penetration tester attempts the highest-ranked paths in the agreed environment, using only techniques the provider's policy allows and avoiding anything the rules of engagement exclude, such as denial of service or access to tenants the client does not own. Testing against production data is limited to what the rules of engagement permit. The tester records each attempt, including failed ones, so the client can see which controls worked. The step is done when every ranked path is marked as confirmed, blocked or not tested, with evidence attached for each result.
Report findings and agree remediation owners
The provider writes a report that separates configuration findings from confirmed attack paths, so the client can see which issues were proven exploitable and which are hygiene gaps. Each item receives a recommended fix, a responsible team and a retest method. The client's engineering and security leads decide the order of work, since fixes to identity policies can affect production workloads. The step is done when the report has been accepted by the client, and every open item has an owner who has agreed to the fix plan.
Remediate and recheck the configuration
The client's platform team applies the fixes, and the reviewer reruns the same checks against the changed accounts to confirm each finding is closed. Where a fix was made in a different way than recommended, the reviewer confirms the new setting meets the same requirement. The recheck should use the same method as the baseline so the results are comparable. The step is done when each closed item has a before and after record, and any item that cannot be closed has a documented risk acceptance signed by the client.
Validate detections against the tested techniques
CIS Safeguard 18.4 asks for security measures to be validated after each penetration test so that detections cover the techniques the testers used. The client's security operations team checks whether logs and alerts fired for each confirmed path, and the provider supplies timestamps and technique descriptions for that comparison. Gaps become detection engineering tickets. The step is done when every confirmed path has either an alert that fired, a logged gap with an owner, or an accepted reason for not alerting.
Retest and schedule the next cycle
The provider retests the paths that were confirmed earlier, and the reviewer repeats the configuration baseline after significant platform changes. The client decides how often these cycles run, since the rules of engagement and the scope may need updates as architecture changes. Confirm with the provider how retests are priced, which is usually part of the original written scope or a separate quote. The step is done when the retest results are filed, the scope has been reviewed, and the next assessment has a date and an owner.
What drives the cost and effort of each approach
DeployOpen does not publish prices, and quotes follow the written scope. The drivers below show what moves the effort for a configuration review and for a cloud penetration test, and why one-time assessments differ from recurring reviews.
Accounts, regions and services in scope
Configuration review effort grows with each account, subscription or project, each region enabled and each managed service switched on. A reviewer checks identity, logging, networking and storage settings in every one of them, so a multi-account estate costs more than a single landing zone even when the benchmark is the same. Ask the provider to list the accounts, regions and services in the written scope and to state how new accounts added during the engagement are handled.
Benchmark profile and control count
The benchmark profile chosen sets how many checks run and how deep each one goes. A level 1 style profile covers baseline settings, while stricter profiles add controls that need more evidence and more judgment. Confirm which benchmark version and profile the quote assumes, and whether the assessor will map findings to the control IDs your auditors use.
Manual versus automated checks
NIST SP 800-115 notes that automated configuration reviews are faster than manual ones, but some settings still need a person to read them in context. Automated tooling drives the bulk of the work, and the manual share, such as reviewing IAM policies, key handling and organizational guardrails, drives most of the senior reviewer time. Ask the quote to separate tool-run checks from manual validation so you can see where the cost sits.
Privileged read access setup
Both automated and manual reviews need root or administrator level read access to view selected security settings, as NIST SP 800-115 points out. Creating a read-only audit role, approving it through your change process and removing it at the end takes coordination time from your cloud and security teams. Ask who creates and tests the role, how access is logged and how it is revoked when the review closes.
Depth of attack paths
A penetration test is priced by how far the tester is asked to chain weaknesses. Reaching a single workload is less work than tracing a path from an exposed application through a role assumption to a data store. Each extra path adds validation, exploitation control and cleanup. Define the target systems and the objective, such as access to a named data set, in the written scope.
Provider notice and prohibited tests
Provider rules change what the tester can run and when. AWS requires 2 weeks' notice for simulated events such as red team exercises, and its policy bars DNS zone walking and flooding tests. Google Cloud does not require notice for testing your own infrastructure, but the test must stay inside your own projects. Ask the provider to confirm which notice applies, and budget time for planning and any scheduling constraints.
Reporting depth and retest rounds
A short findings list costs less than a report that documents each attack path with evidence, business impact and remediation steps. Retest rounds, which confirm that fixes work, are a separate line item and recur with each cycle. Ask for the report format, the number of retest rounds included and whether a debrief with your engineers is part of the fixed scope. Configuration reviews also repeat, so ask whether the quote covers a one-time assessment or a recurring review cycle with a set cadence.
Questions buyers ask before choosing a cloud assessment
What does a cloud configuration review or cloud penetration test cost?
No supplied figure sets a price, and DeployOpen does not publish one. Cost follows from the written scope: the number of cloud accounts, subscriptions and projects, how many regions and services are in play, whether identity and hybrid links are included, and how much of the work is automated checking versus manual validation. A configuration review is mostly analyst time spent checking settings against a benchmark and interpreting results, while a penetration test adds time for attack paths, validation and retesting. Ask for a quote that separates the two, lists the accounts and services covered, states what is one-time (the assessment and report) and what recurs (rescans, retainers or periodic retests), and names the provider rules the tester will follow.
How long does a cloud assessment take, and what sets the timeline?
The duration depends on scope size and how quickly access and answers arrive, so no fixed timeline applies across engagements. The main drivers are the number of accounts and workloads, the depth of identity review, the time needed to agree rules of engagement and notification steps, and the effort to validate findings and retest fixes. Penetration tests usually take longer than configuration reviews because they include attack-path work and validation. Confirm in the written scope the start date, the testing window, the reporting milestones and what happens if access is delayed. AWS, for example, asks for two weeks' notice before simulated events such as red team exercises, which can set the earliest start for those exercises.
What access and internal staff time does a cloud assessment need?
A configuration review needs read-only or audit-level access to the cloud control plane, such as configuration, logging and identity settings, and NIST SP 800-115 notes that both automated and manual configuration reviews need root or administrator privileges to view selected security settings. A penetration test needs a defined starting point, such as a standard user or workload identity, and often a sandbox or agreed production path. Internal staff time goes to providing accounts, approving scope, answering questions about architecture, reviewing findings and running fixes. Ask the provider which roles and credentials it needs, how they are issued and revoked, and who on your side owns each approval.
Can our internal team run a cloud configuration review without outside help?
Often yes, for the configuration side, if the team has people who know the cloud provider's settings and the chosen benchmark. Automated tools can check many settings quickly, and NIST SP 800-115 says automated reviews are faster than manual ones, but some settings still need manual checks. Internal teams find it harder to be independent of the systems they run, to judge attack paths across accounts and identities, and to produce evidence an assessor will accept. The CIS Controls ask for penetration tests at least annually (Safeguards 18.2 and 18.5), so a self-run review does not replace that test. Confirm what your auditor or customer expects before deciding who performs the work.
Does a cloud penetration test need notice to the cloud provider?
It depends on the provider, so check its current policy before testing. Google Cloud states that customers do not need to notify Google before penetration testing their own Google Cloud infrastructure, but they must follow the Acceptable Use Policy and Terms of Service and affect only their own projects. AWS prohibits activities such as DNS zone walking, denial of service, port or request flooding and S3 bucket or subdomain takeover, and requires two weeks' notice for simulated events such as red team exercises. Microsoft's rules of engagement prohibit DDoS and network-intensive fuzzing and require vulnerabilities in Microsoft services to be reported to MSRC. Confirm in the quote who gives notice, when, and which tests are excluded.
Does PCI DSS require a penetration test, and does a configuration scan satisfy it?
PCI SSC's Penetration Testing Guidance (v1.1, September 2017) describes penetration testing as essentially a manual process and says that simply running an automated tool does not satisfy the penetration testing requirement. A configuration review is therefore useful evidence of cloud hygiene but does not, on its own, meet a penetration testing requirement. Ask your assessor which requirement applies to your cardholder environment and whether the in-scope cloud components need both a review and a manual test. This is context for scoping, not a statement of how your assessment will be judged.
Which should we buy first, a configuration review or a penetration test?
For most teams, a configuration review comes first because it establishes a baseline and removes obvious misconfigurations that would waste penetration testing time. A penetration test then tries to chain weaknesses across identities, workloads and network paths that a settings check cannot prove. NIST SP 800-115 notes that organizations often combine techniques to reach an in-depth assessment at an acceptable level of risk. Ask the provider how it will hand findings from the review into the test scope and whether retesting of fixes is included in the quote.
When compliance frameworks ask for each approach
Compliance frameworks rarely ask for a cloud configuration review or a cloud penetration test by name, but they set cadences that decide how often each one must happen. Start by identifying which requirement applies to you, then match the cadence to the method. CIS Controls v8.1 Safeguards 18.2 and 18.5 ask for external and internal penetration tests at least annually, which points toward attack-based work on cloud-hosted workloads and identities. Safeguard 18.4 adds that security measures be validated after each test so that detections cover the techniques the testers used. Ask your assessor whether they expect the penetration test evidence to cover your cloud estate or only your on-premises network.
For EU financial entities, DORA Article 25(1) lists vulnerability assessments and scans alongside penetration testing, so both methods can be needed under one regime. Commission Delegated Regulation (EU) 2024/1774 goes further for ICT risk management: Article 10 requires automated vulnerability scanning of assets supporting critical or important functions at least weekly, and Article 21 requires access rights on those systems to be reviewed at least every 6 months. The weekly scan and the six-monthly access review are configuration and identity work, so a cloud configuration review, scoped to the accounts, IAM roles and key policies that support those functions, is the more natural evidence source. Confirm with your competent authority whether a configuration review alone satisfies your scanning and access review obligations.
NIST SP 800-53 Rev. 5 control CA-8 Penetration Testing, with its enhancements for an independent team and red team exercises, is selected in SP 800-53B only in the high baseline. If your system is categorized at a lower baseline, CA-8 may not be a control you are required to implement, so check your system security plan before budgeting for it. If you are at the high baseline, plan the penetration test as a separate deliverable from the configuration review, and record which cloud services, accounts and identities were in scope. Confirm the applicable baseline and any agency or sector overlay with your assessor or regulator before you commit to a scope.
How DeployOpen scopes and delivers cloud security assessments
DeployOpen plans, implements, tests and documents self-hosted open-source platforms inside the client's own cloud accounts, and runs security testing alongside that work. For cloud assessments, we start from a written scope that separates the configuration review of accounts and identities from any penetration test of workloads, and we agree the provider rules that apply before any testing begins.
How we scope
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
- The written scope names which cloud accounts, subscriptions or projects are in the review, and which are excluded.
- We record which access the review needs, such as read-only roles for configuration checks or elevated roles for privilege testing.
- The scope confirms which provider testing policy applies and which prohibited activities are off the table for the engagement.
How we build
- We run configuration checks against the benchmark the client selects, and record which settings need manual verification.
- Penetration testing covers the workloads and identities named in scope, using only the techniques the provider policy permits.
- Findings are tied to the account, resource or identity where they were observed, so the client can reproduce each one.
- Where a test touches provider services, we follow the notice and reporting duties set out in the rules of engagement.
How we hand over
- Deliverables include the scope record, the test log, the findings with evidence, and the remediation steps for each one.
- We document which checks passed, which failed and which could not be completed, so the gaps are visible to the client.
- Results are separated into configuration findings and attack-path findings, so each can be assigned to the right owner.
- We recommend a re-check of the fixed items, and we confirm with the client what evidence an assessor will need to see.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, separate from the initial fixed-scope work.
- A retainer can schedule recurring configuration reviews, so new accounts and resources are checked as the environment changes.
- Ask for a written quote that lists scope, access requirements, provider notice steps and deliverables before you commit.
- Confirm with DeployOpen how each recurring check is scoped, priced and reported before the retainer is signed.
