Which open-source tools check systems against CIS Benchmarks?
Wazuh SCA, OpenSCAP, Fleet, Kyverno and the free CIS-CAT Lite can each produce pass or fail results against Benchmark-based checks, and CIS Benchmark PDFs are free to download. When a customer, contract or regulator asks for evidence that systems follow a CIS Benchmark, the question is usually whether free tooling can produce that evidence at all. For most estates it can, provided the checks are matched to the right asset type and the results are kept. The organization owns its configuration baseline and the evidence it produces. An implementation partner can prepare the scanning, tune the policies, schedule the runs and package the results. Only an independent auditor or assessor judges whether the configuration conforms.
Tool choice follows the asset type. Wazuh Security Configuration Assessment (SCA) suits endpoints and Linux or Windows hosts managed by a Wazuh agent, using YAML policies that are shipped mostly based on CIS Benchmarks. OpenSCAP's oscap scanner suits hosts that need SCAP content such as SCAP Security Guide profiles, with HTML, ARF or XCCDF output. Fleet policies suit laptops and servers already enrolled through osquery, where each policy is a yes or no query per device. Kyverno policy reports suit Kubernetes clusters, because they record results from admission events and background scans of existing resources. CIS-CAT Lite covers select technologies and returns a compliance score, so it works best as a spot check beside the continuous tools.
Before you rely on any of these, confirm three things with your own team. First, that the Benchmark version and profile your policy uses matches what your baseline document names, since CIS revises Benchmarks over time. Second, which checks the tool cannot run, and how you will cover those gaps by hand. Third, whether your auditor expects results from CIS-CAT or accepts equivalent scanner output. Keep the raw reports with timestamps and the remediation tickets for failures. Those records, not the tool's dashboard, are what a later review will ask to see.
What each open-source scanner reports
Each tool below checks systems against CIS Benchmarks or secure configuration baselines, but they report different things in different formats. Read the output format and the coverage limits before deciding which scanner produces the evidence your Safeguard 4.1 process needs.
Wazuh SCA policies
Wazuh Security Configuration Assessment scans monitored endpoints against YAML policies, most of them based on CIS Benchmarks and extendable with your own. Checks cover files, directories, registry keys and running processes, and each failed check comes with remediation advice. Confirm which Benchmark versions ship with your Wazuh release, and check whether your custom policies still map to the Benchmark sections your auditor will sample.
OpenSCAP oscap reports
The oscap scanner evaluates a system against SCAP content in XCCDF and OVAL, such as profiles from the SCAP Security Guide. It reports a pass or fail for each rule and writes results as HTML, ARF or XCCDF. The ARF and XCCDF files suit evidence archives, while the HTML report is easier for a reviewer to read. Profiles from the SCAP Security Guide may not match a CIS Benchmark section for section, so confirm the mapping before you cite it.
Fleet policy results
Fleet policies are yes or no questions asked of devices as osquery queries. The policies page shows which devices in each fleet pass or fail each policy, which gives you a current, device-level view of configuration drift. Fleet policies check only what osquery can query, so a Benchmark recommendation that needs a file read, a service check or a kernel parameter must be written as a query before it appears here. Confirm query coverage against each Benchmark section you plan to rely on.
Kyverno policy reports
Kyverno writes PolicyReport and ClusterPolicyReport resources with pass, fail, warn, skip or error results for each matching resource. Results come from admission events and periodic background scans of existing resources, so the report reflects current cluster state. Kyverno covers Kubernetes objects only, not operating system settings on nodes or cloud accounts. Check which policies you have written or adopted, since a report shows only the rules that were deployed.
CIS-CAT Lite scores
CIS-CAT Lite is free and allows unlimited scans of the configuration of select technologies against select CIS Benchmarks, producing a compliance score. It is the only tool here built by CIS itself, so its rule logic follows the Benchmark directly. Its coverage is limited to the technologies and Benchmarks it supports, and tailoring the profile to your environment requires a SecureSuite membership. Confirm your target technologies are in scope before you plan a deployment.
What a scan result proves for Safeguard 4.1
A scan result proves one thing well: the state of a system at the moment it was checked against a rule set. It does not prove that Safeguard 4.1 is met. CIS Control 4 Safeguard 4.1 asks the organization to establish and maintain a secure configuration process. The CIS Controls Assessment Specification scores that Safeguard as failing when no such process exists, or when the process was last reviewed and updated more than 12 months ago. Its metric is standard configuration coverage, meaning how much of the in-scope estate is held to a documented standard. A Wazuh SCA dashboard, an oscap HTML report, a Fleet policy page or a Kyverno PolicyReport can all show coverage, but only against the baseline you chose and the assets you pointed them at.
The evidence an assessor will ask for sits outside the scanner. Keep the written secure configuration process, the owner who approves baseline changes, the date of the last review and the record of that review, the list of approved deviations with their justifications and expiry dates, and the scope statement that says which assets the scan covers. Export the scan output with a timestamp and the policy or profile version used, so the result can be tied to a specific review cycle. Confirm with your assessor which of these artifacts they expect in the format they expect, because the Assessment Specification describes what to measure and leaves the how to the organization.
Be explicit about the limits of this evidence in any document you give a customer, regulator or auditor. CIS states that the Controls and Benchmarks do not replace any regulatory, compliance or authorization scheme. A passing scan on a CIS Benchmark profile is therefore an input to a broader program, not a conformity claim for a standard that names its own requirements. If a contract or regulator asks for secure configuration, identify the specific requirement it cites and confirm with the auditor whether a Benchmark scan result is accepted as evidence for it. Only an independent assessor can state that a control is met.
Which tool fits which environment?
Servers and workstations running Linux or Windows are where most CIS Benchmark checks apply, and two open-source options cover them well. Wazuh's Security Configuration Assessment runs YAML policies on monitored agents, most of them based on CIS Benchmarks, and checks files, directories, registry keys and running processes. OpenSCAP's oscap scanner evaluates a system against SCAP content such as SCAP Security Guide profiles and reports pass or fail for each rule.
- Wazuh SCA output: Per-agent pass, fail and not-applicable results with remediation text, collected centrally in the Wazuh dashboard and queryable over time.
- OpenSCAP output: HTML, ARF or XCCDF reports per scan, which suit point-in-time evidence packs for a specific host build.
- Policy coverage: Confirm which Benchmark version and profile each policy implements, since shipped content may not match the current CIS release.
- Tailoring: Document every rule you disable or change, because an unexplained deviation reads as a gap during review.
- The gap: Neither tool approves a secure configuration process, sets the baseline owner, or records exceptions with review dates. Your team must write those records and show that the scan results feed them.
Before relying on either tool's output, confirm with the assessor which Benchmark edition and profile they expect for your systems.
Fleet's policies ask yes or no questions of enrolled devices using osquery queries, and its policies page shows which devices in each fleet pass or fail each one. This model fits laptops, desktops and mixed endpoint estates where you need a continuously refreshed answer to questions like whether disk encryption is on or a host firewall is enabled. Fleet is MIT licensed, so the platform itself carries no license cost.
- Fleet policy output: A pass or fail status per device and per policy, visible on the policies page and exportable for review.
- Query-based checks: Each policy is a single osquery question, so checks reflect what the device reports at query time rather than a full Benchmark rule set.
- Coverage limits: Translating every CIS Benchmark recommendation into a yes or no query is laborious, so expect partial coverage of any single Benchmark.
- Change history: Keep dated snapshots of policy results, because a live dashboard alone may not satisfy an assessor asking what the state was at a past date.
- The gap: Fleet does not map its policies to CIS Safeguards or a specific Benchmark profile. You must build that crosswalk, define the remediation owner for each failing policy, and document exceptions.
Ask your assessor whether query-based policy results are accepted as configuration evidence for the assets you plan to cover.
Kyverno is a policy engine for Kubernetes, and its PolicyReport and ClusterPolicyReport resources record pass, fail, warn, skip or error results for each matching resource. Results come from admission events as resources are created or changed and from periodic background scans of existing resources, so the report reflects current cluster state. Kyverno is Apache-2.0 licensed.
- Report output: Structured Kubernetes resources that you can query with standard tooling or export to a dashboard.
- Admission and background results: Admission events catch new non-compliant resources, while background scans catch drift in what already runs.
- Scope: Kyverno checks resource policies such as pod security settings and image rules. It does not evaluate full host Benchmarks on the nodes beneath the cluster.
- Node hardening: Use a host-level tool such as Wazuh SCA or OpenSCAP on the worker and control plane nodes, and treat cluster policy results as one layer of the evidence.
- The gap: Kyverno reports do not show whether your cluster configuration matches a named CIS Kubernetes Benchmark profile. Map each policy to the requirement it supports and record why any policy is set to audit rather than enforce.
Confirm with your assessor whether resource policy reports, node scan results and a documented baseline together meet the secure configuration evidence they expect.
CIS-CAT Lite is CIS's free assessor tool. It is free, offers unlimited scans and evaluates the configuration of select technologies against select CIS Benchmarks, giving a compliance score. For one host, a handful of machines, or a pilot before a wider rollout, it gives a quick, Benchmark-named result without standing up a management platform.
- Output: A compliance score and a per-recommendation result for the technologies it supports.
- Coverage: Only select technologies and Benchmarks are included, so check the supported list before planning around it.
- Repeatability: Run it on a fixed schedule and save each report, since a single scan shows state only at that moment.
- Pro features: Tailored content, reports mapped to the CIS Controls and centralized management require CIS-CAT Pro and a SecureSuite membership.
- The gap: A small estate still needs a written secure configuration process, an owner for each baseline and a record of deviations. Lite does not provide those, and it does not manage results across many systems.
Before you rely on CIS-CAT Lite for evidence, confirm with your assessor that a manually run, saved report is acceptable for the systems in scope.
How a team rolls out CIS Benchmark scanning
The sequence moves from choosing the right Benchmark and profile for each asset class to a recurring scan schedule with dated reports. Each step produces an artifact an assessor can inspect, so the team can show what was configured, what failed and what was accepted.
Confirm the Benchmark and profile for each asset class
Start by listing the asset classes in scope, such as Linux servers, Windows endpoints, Kubernetes clusters and cloud accounts, and matching each to a CIS Benchmark that covers its operating system or platform. Decide the profile for each one, Level 1 or Level 2, or a STIG profile where a federal or contractual requirement applies. Confirm the Benchmark version in use, since CIS Benchmark PDFs are free to download but tailoring recommendations need a SecureSuite membership. The output is a short table of asset class, Benchmark, version and profile, approved by the owner of the configuration process.
Write the secure configuration process
Document the secure configuration process the organization will follow, covering who approves a Benchmark profile, how changes are tested, how deviations are recorded and how often the process is reviewed. This is the document Safeguard 4.1 in the CIS Controls assesses, and the assessment specification scores it as failing if no process exists or it was last reviewed and updated more than 12 months ago. Close the step when the process has a named owner, an approval record and a review date.
Test Level 2 and tailored profiles before rollout
Before any Level 2 profile or tailored setting reaches production, test it on non-production systems that match the real builds. Level 2 profiles aim at higher security and can change how services, logging or remote access behave, so the operational risk needs to be understood first. Record which settings broke which workloads and whether a fix or a documented exception was needed. The output is a test record that links each proposed change to its result.
Deploy the scanner to a pilot group
Install the chosen scanner on a small pilot group that represents each asset class, for example Wazuh SCA agents on a set of Linux and Windows hosts, OpenSCAP oscap runs on a sample of RHEL-family servers, or Fleet osquery policies on a subset of devices. Confirm the scanner reads the right policy or profile and that results return to the central console. Pilot results should be checked against a manual spot check on at least one host so the team trusts the tool before scaling it. Close the step when every pilot asset reports a result.
Record the baseline results
Run the full scan across the pilot group and save the results as the baseline. Export the pass and fail counts per rule, such as Wazuh SCA results, OpenSCAP HTML, ARF or XCCDF reports, Fleet policy pass and fail counts per device, or Kyverno PolicyReport entries for cluster resources. Store the files with the scan date, the Benchmark version and the policy or profile identifier. The baseline is the starting point against which later progress is measured, so it should be kept unchanged after it is recorded.
Triage failures into fixes, deviations or accepted risk
Review each failing rule and sort it into one of three outcomes: a fix that the team applies and re-tests, an approved deviation recorded with a business reason and an owner, or an accepted risk signed off by someone with authority to accept it. Scanners report failures but cannot judge whether a failure matters in a given environment, so the decision and its approver must be written down. Close the step when every failing rule has one recorded outcome and the fixes have been re-scanned to confirm they pass.
Schedule recurring scans and expand coverage
Set a recurring scan schedule for each asset class and expand the scanner from the pilot group to the full inventory once results are stable. Recurring scans matter because configuration drifts after build, so a host that passed in the baseline can fail later. Confirm that new assets are picked up by the scanner automatically or added through the asset inventory. The output is a documented schedule and a coverage report showing which assets were scanned on which dates.
Retain reports as dated evidence
Keep each scan report, the triage record, the deviation approvals and the change records together in a location with access control and retention rules. Dated reports let an assessor compare the baseline with later scans and check that failures were handled. Ask the independent auditor which evidence format and retention period it expects before the assessment begins, because the organization owns its controls and the auditor decides what is sufficient. Close the step when a sample of reports can be retrieved by date and asset.
Where free tools fall short
The free tools in this topic give a real starting point, but their results are only as current and complete as the Benchmark content behind them, and a passing score does not by itself show that a system matches the Benchmark version your assessor expects. Before relying on any tool, confirm which CIS Benchmark versions and profiles it covers, because Wazuh SCA policies, OpenSCAP SCAP content and CIS-CAT Lite evaluate different sets of technologies on different update cycles. Check each tool's current coverage with its vendor or documentation, and record the Benchmark version and profile used for every scan so that results can be matched to the secure configuration process under Safeguard 4.1.
Tailoring is the next gap. CIS Benchmark PDFs are free, but the machine-readable formats that make tailoring and automated import practical, such as XCCDF, Word, Excel and XML, come through CIS WorkBench, and tailoring recommendations require a SecureSuite membership. Without that access, your team must edit profiles by hand, and every deviation from a Benchmark recommendation needs a documented reason. Expect false positives where a setting is correct for your environment but the rule assumes a default, such as a service account that legitimately needs a different login policy. Each exception should be reviewed and approved, not silently suppressed in the tool.
Coverage and credentials decide whether a scan means anything. Agent-based checks such as Wazuh SCA see only endpoints where the agent runs, and policy-based checks such as Fleet queries or Kyverno reports cover only what the device or cluster state exposes. Many Benchmark checks need privileged or root access, so a scan run with limited credentials can pass silently on settings it never read. Ask each tool which checks were skipped or errored, not just which passed. Finally, budget for reporting work: none of these tools produces a report mapped to CIS Controls Safeguards by default, so someone must build the mapping, schedule exports and keep it current as Benchmarks change.
What usually breaks open-source scanning programs
A scanner produces results on its own, but it does not produce a secure configuration program. The failures below tend to appear after the first successful scan, when nobody has decided who acts on the results, how exceptions are approved, or how evidence is kept for an assessor.
Scanning without a written process
Teams often deploy Wazuh SCA, OpenSCAP or CIS-CAT Lite and start reading results before anyone has written down which Benchmark and profile applies to which systems. CIS Safeguard 4.1 expects an established and maintained secure configuration process, and an assessor will ask for it before looking at scan output. Check that the process names the Benchmarks in use, the owner of each asset class, and the review cycle.
Unowned failures that never close
A failing rule sitting in a dashboard is not a remediation decision. Many programs see the same findings week after week because no engineer or team is named for each failing check, and no ticket tracks the fix. Confirm that every recurring failure maps to a named owner, a target date set by that owner, and a closure record.
Deviations left undocumented
Some Benchmark recommendations break a legitimate application or a vendor requirement. When teams silently disable a rule or leave it failing, the scan score no longer reflects the actual risk. Each accepted deviation should record the rule, the affected systems, the reason, the compensating control and an approver who is not the person requesting the exception.
Drift between scans and Benchmark versions
A scan result is only meaningful against the Benchmark version and profile it was run with. When a Benchmark is revised, or when a policy file is updated in one tool but not in another, the same host can pass in one report and fail in another. Record the tool version, policy or content version and Benchmark version alongside each run, and rerun after major content changes.
Alerts with no owner in the security team
Scanner alerts often flow to a shared mailbox or channel where nobody is accountable for triage. Platform teams see configuration noise, while security teams assume someone else is reviewing it. Assign a triage role inside the security function, set severity rules for which failures page someone, and review unacknowledged alerts in the same meeting as other control findings.
Reports kept only in the tool
Dashboards are useful for operations but weak as audit evidence. If the only copy of a compliance result lives in the scanner database, it may be lost on upgrade, purged by a retention setting, or unavailable when an auditor asks for the state of a system at a past date. Export dated HTML, XCCDF or ARF reports to a retained evidence store, and confirm the retention period in each tool's configuration.
CIS Benchmark scanning questions buyers ask
Can free CIS Benchmark scanners satisfy an auditor?
Free scanners produce evidence, but the auditor decides whether that evidence satisfies a requirement. An assessor will look at which Benchmark and profile you chose, whether the scan covers the systems in scope, how often it runs, and what happened to every failed rule, including documented deviations. Your organization owns the secure configuration process under Safeguard 4.1, and the CIS Controls Assessment Specification measures whether that process exists and was reviewed within the last 12 months. A scanner report is one input to that case. Confirm with your auditor, before you rely on a tool, which report formats and retention periods they expect.
What does a free CIS Benchmark scanner cost to run?
The scanner licenses are free, so the real costs are people, hosting and remediation. Staff time covers choosing profiles, writing or tailoring policies, triaging failed rules, documenting deviations and reviewing results. Hosting covers the management server, storage for scan results and retention of logs. Both costs grow with estate size, the number of operating systems and cloud accounts, the number of scan cycles and how many exceptions need review. Ask any consultant for a quote that separates one-time setup (tool deployment, baseline selection, first policy tuning) from recurring work (scan operation, reporting and review). DeployOpen quotes follow from a written scope.
Do we need a CIS SecureSuite membership to scan?
No. CIS Benchmark PDFs are free to download, and the free CIS-CAT Lite tool scans select technologies against select Benchmarks. SecureSuite membership matters for other needs. Members can download formats such as XCCDF, Word, Excel and XML through CIS WorkBench, and tailoring recommendations require membership. CIS-CAT Pro also requires SecureSuite membership. If your team needs to adjust a Benchmark to your environment in a way the free PDFs do not support, confirm the current membership terms and pricing with CIS directly, since those terms are not set out in the sources used for this page.
Which open-source CIS scanner should we start with?
Start with the asset types that carry the most risk and check which tool already covers them.
- Linux and Windows servers and endpoints: Wazuh Security Configuration Assessment runs YAML policies, mostly based on CIS Benchmarks, across monitored endpoints and recommends remediation.
- Red Hat-style hosts and SCAP-based compliance: OpenSCAP's oscap scanner evaluates systems against SCAP content such as SCAP Security Guide profiles and writes HTML, ARF or XCCDF reports.
- Laptops and workstations: Fleet policies ask yes or no osquery questions and show which devices pass or fail each policy.
- Kubernetes clusters: Kyverno writes PolicyReport resources with pass, fail, warn, skip or error results from admission events and background scans.
Confirm that each tool's content matches the exact Benchmark version and profile you select before you rely on the results.
How often should we run CIS Benchmark scans?
Scan on a recurring schedule tied to your secure configuration review cycle, and scan again after any significant change to a build, image or cloud account. The CIS Controls Assessment Specification fails Safeguard 4.1 when the secure configuration process was last reviewed and updated more than 12 months ago, so the review cycle sets a floor. Between formal reviews, continuous or periodic scans show drift from the baseline. Agree the schedule, the owner of each failed rule and the deviation approval path before the first scan, and keep the results so an assessor can see the trend over time. Your scan frequency should be a documented decision, not a default.
Do we need a consultant to set up CIS Benchmark scanning?
You do not need one, but many teams bring in outside help to build the process, deploy the scanners and prepare the evidence. A fixed-scope engagement with DeployOpen covers implementing and configuring the scanners inside your own infrastructure, tuning policies to your chosen profiles, and documenting the process and evidence trail. DeployOpen prepares systems and evidence but does not issue audit reports or certifications, and your organization still owns its controls and the decisions behind them. Only an independent auditor or certification body issues assessment outcomes. Ask any provider for a written scope, deliverables and timeline before work starts.
Who owns the scanning program and who only prepares it
The organization owns the scanning program. Its leadership decides which Benchmark and profile apply to each platform, which findings count as failures, how remediation is prioritized, and who approves a deviation. A scanner reports what the configuration looks like on a given day. It does not decide whether that configuration is acceptable for the business. Those decisions belong in the organization's own secure configuration process, and they should be signed off by someone who answers for the risk, not by the team that installed the tool.
An implementation partner such as DeployOpen can plan, deploy, customize, harden and document the scanners and the evidence pipeline inside the client's own cloud accounts, data centers or air-gapped networks. In practice that can include standing up Wazuh SCA, OpenSCAP, Fleet or Kyverno, tailoring policies with the client, scheduling scans, routing results to a retained store, and writing the runbooks that show how a failing check moves through triage to a fix or an approved exception. Scope, deliverables and timelines should be agreed in writing before work starts, and the client should confirm whether it needs tailored Benchmark content, which requires a CIS SecureSuite membership.
Only an independent auditor, assessor or certification body issues reports or certificates. DeployOpen does not issue audit reports, attestations or certifications, and a passing scan report from an open-source tool is evidence for an assessor to review, not a conformity statement in itself. Before engaging anyone, ask the assessor which scan outputs, retention periods and exception records it expects, and ask your implementation partner to show how each of those artifacts is produced and kept.
How DeployOpen deploys CIS Benchmark scanning
DeployOpen plans, implements and hardens self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks, including the scanning tools that check systems against CIS Benchmarks and secure configuration baselines. We prepare the scanning and the evidence it produces, while your organization owns its controls and an independent assessor decides whether they meet the standard.
How we scope
- We agree a written scope covering which platforms, operating systems and cloud accounts are scanned before any work starts.
- The scope names which Benchmarks and profiles apply, and which deviations your team has approved and will record.
- Engagements are fixed-scope, so deliverables and timelines are confirmed in writing before implementation begins.
How we build
- We deploy the scanning tools in your environment, such as Wazuh SCA, OpenSCAP, Fleet or Kyverno, matched to the assets in scope.
- We configure policies and tailoring against the agreed Benchmark profile and test them on representative systems first.
- We check that scan results reach the reporting dashboards and that failed checks route to the people who own remediation.
How we hand over
- You receive documentation of the scanning architecture, policy versions, scan schedules and how each report maps to Safeguard 4.1 evidence.
- We walk your operations and security staff through running scans, reading results and exporting reports for review.
- Evidence exports are organized so your internal team or an independent assessor can trace each finding to its source.
How we support
- Ongoing support, upgrades and continuous testing of the scanning stack are available as a retainer, agreed in writing.
- Retainers can cover updating policies when CIS publishes new Benchmark versions, after the agreed scope is reviewed.
- Confirm with us during scoping which support level, response arrangements and reporting cadence your organization needs.
