Which open-source tools produce NIST 800-53 evidence, and what they cannot do
When a contract, agency or regulator requires NIST SP 800-53, self-hosted open-source tools can generate much of the configuration, logging, integrity, scanning, MFA and secrets evidence that specific controls call for. The organization still owns its controls, and only an independent assessor issues the conclusion on whether they are met. Treat the tools as producers of evidence for the assessor to test, not as proof that the system is compliant. How far the tools go depends on three things: the system's impact level, the baseline selected under SP 800-53B, and the tools the organization already runs. A low-impact system needs a smaller set of controls than a high-impact one, so the same tool covers a larger share of the work in some cases than in others. Confirm the baseline and the control version first, since the Release 5.2.0 changes added SA-15(13), SA-24 and SI-2(7) and revised SI-7(12), and any new enhancement needs its own evidence source. Map each selected control to a named tool, a named owner and an evidence artifact before any installation begins.
Several tools map cleanly to specific controls. OpenSCAP evaluates systems against SCAP content from ComplianceAsCode and produces HTML, ARF or XCCDF reports that support configuration evidence for CM-6 and related settings. Wazuh supplies log collection and file integrity monitoring records that help with audit and integrity controls, and Greenbone (OpenVAS) produces vulnerability scan results relevant to risk assessment and flaw remediation. Keycloak supports passkeys, recovery codes and TOTP or HOTP for multi-factor authentication, and OpenBao handles secrets and short-lived credentials. Each tool can show that something happened or was configured, but none of them decides whether a control is satisfied under the assessment procedures in *SP 800-53A*. The gaps matter as much as the coverage. Scanners do not cover policies, training records, personnel screening, physical controls or risk decisions, and tool output does not replace the system security and privacy plan or the plan of action and milestones. Ask your assessor which artifacts they will accept, confirm each tool's current capabilities against its own documentation, and budget for the internal effort of keeping configurations, retention and access reviews current. Where the operating model is unclear, a fixed-scope implementation partner can prepare the systems and evidence, while the assessor remains independent.
Mapping each open-source tool to the SP 800-53 evidence it produces
Each tool below produces machine-generated records that can support specific SP 800-53 control families. None of them satisfies a control on its own, so the organization still has to show that the output meets the assessment procedure for each control in SP 800-53A and that the configuration, scope and retention match its own system.
OpenSCAP for configuration checks
OpenSCAP is an LGPL-2.1-or-later scanner that evaluates a system against SCAP content in XCCDF and OVAL, reports pass or fail for each rule, and writes HTML, ARF or XCCDF results. Those results are candidate evidence for CM-6 (configuration settings). Before relying on them, confirm which profile was applied, that it matches the system's documented baseline, and that the assessor's procedure for CM-6 accepts scan output as an examine method.
ComplianceAsCode content and remediation
ComplianceAsCode began in 2011 as the SCAP Security Guide and was renamed in September 2018. It produces SCAP source data streams, Ansible playbooks, Bash scripts and CEL rules for RHEL, Fedora, Ubuntu, Debian, SUSE, and Kubernetes and OpenShift through the compliance-operator. The content supports CM-6 and CM-2 evidence, but the organization must confirm which rules it selected, record any deviations as tailoring decisions, and keep the content version tied to each assessment.
Wazuh log collection for audit records
Wazuh is a GPL-2.0-only platform whose agents forward system and application logs to a central server, which can also receive syslog and third-party API data. It analyzes events through decoding and rule matching. This output supports AU-2, AU-3, AU-6 and AU-12 evidence. Confirm which event sources are in scope, how long records are retained, and whether the log store meets the integrity and protection expectations in the AU procedures.
Wazuh file integrity monitoring
Wazuh file integrity monitoring records a baseline of checksums and attributes for monitored files, then alerts when a user or process creates, modifies or deletes them, either on a schedule or in real time. The alerts record how and when a change happened and who made it. This output is evidence for SI-7 (software, firmware and information integrity). The team must define which paths are monitored, approve the baseline, and show how each alert was reviewed and resolved.
Greenbone Community Edition scan results
Greenbone Community Edition runs the OpenVAS Scanner under AGPL-3.0-or-later. It executes vulnerability tests from the daily updated Community Feed against targets, controlled by gvmd. Scan reports are inputs to RA-5 (vulnerability monitoring and scanning) and to flaw remediation under SI-2. Confirm the scan frequency and authenticated scan coverage, record the feed version used on each scan, and keep the remediation tickets that close each finding, since a scan report alone does not show remediation.
Keycloak authentication evidence
Keycloak is an Apache-2.0 identity server that supports two-factor authentication with passkeys, recovery codes and TOTP or HOTP codes, along with configurable one-time password policies and W3C WebAuthn settings. Its realm configuration and login event logs support IA-2 (identification and authentication for organizational users) and IA-5 (authenticator management). Confirm which authenticator types are permitted, how recovery codes are issued and stored, and how the realm settings are exported as evidence for the assessor.
OpenBao secrets and key lifecycle
OpenBao is an MPL-2.0 secrets manager that stores secrets encrypted in one place, issues dynamic short-lived credentials for systems such as databases, attaches leases that revoke secrets on expiry, and can revoke single secrets or whole credential trees. These mechanisms support SC-12 (cryptographic key establishment and management) and IA-5 evidence. The organization must confirm its key custody model, how unseal and recovery material is held, and whether the cryptographic modules meet any FIPS requirement the system's categorization sets.
Why a tool report is not a control assessment
An OpenSCAP or Wazuh report is evidence for an assessor, not the assessment itself. SP 800-53A gives each control an assessment procedure built from determination statements and three methods: examine, interview and test. Each method is applied to assessment objects such as policies, procedures, people, mechanisms and system behavior. A passing oscap scan of a CM-6 baseline covers the test of configuration state. It does not show that a documented configuration management policy exists, that administrators follow it, or that a change board reviewed the last deviation. For controls such as AT or PS, the tool may contribute almost nothing, and the assessor will rely on interviews and records.
SP 800-37 Rev. 2 encourages automated security and privacy management and reporting tools for building and maintaining authorization packages, and it notes that these tools can significantly reduce documentation costs. It also accepts manual methods for controls that do not suit automation. Practically, this means the organization should map each control in its System Security and Privacy Plan to one of three sources: an automated output from a tool such as Greenbone or Keycloak configuration exports, a manual artifact such as a signed policy or training record, or a combination. Confirm with your assessor which artifacts they will accept for each control before you build the automation, because a tool output that cannot be tied to a named control objective will often be set aside.
NIST does not endorse or offer any compliance tool, and its risk management FAQ says the same about compliance programs. Under FISMA, assessments of agency systems are performed by independent parties, including Inspectors General, and organizations pursuing SP 800-53 for contracts or certification should expect the same independence from the people who configured the tools. DeployOpen can prepare systems, configurations and evidence collection for an assessment, but the organization owns its controls and an independent assessor issues the conclusion. Ask any assessor early which tool outputs they will sample, how they will test the tool's own configuration, and whether they need access to the raw data behind a dashboard.
How the tool stack changes for a cloud, on-premises or air-gapped system
In a cloud account, the tools usually reach their update sources directly over the internet, which makes feed and content updates simple to schedule. The control owner still needs to show that updates are pulled from known sources, that versions are recorded, and that the evidence store sits under the organization's own account controls rather than with the vendor.
- Update sources: OpenSCAP content, ComplianceAsCode releases, Wazuh packages and the Greenbone Community Feed can be pulled from their public sources through a managed egress path, which should be logged and restricted to approved destinations.
- Feed versions: Record the Greenbone feed date and the ComplianceAsCode content version used for each scan or scan baseline so that results can be tied to the content that produced them.
- Evidence storage: Store scan reports, Wazuh alerts and OpenSCAP results in an encrypted bucket or volume in the organization's account, with object lock or retention settings that match the organization's records policy.
- Secrets and keys: Run OpenBao and Keycloak inside the account's private network, with access limited to the workloads that need them.
Before committing, confirm with each vendor whether the cloud deployment path requires outbound internet access, and check whether a mirror or proxy is supported for feeds and packages.
In an on-premises data center, the tools run on hardware the organization controls, usually behind an internal package mirror and a firewall with limited outbound access. Updates become scheduled transfers that someone must stage, verify and approve, and the evidence trail must show that each staged update was reviewed before it reached production.
- Internal mirrors: Mirror operating system packages, container images and tool updates into an internal repository, and record the mirror sync date alongside each update.
- Staged feed updates: Download the Greenbone Community Feed and ComplianceAsCode content on a controlled host, check the content version, then distribute it to scanners and assessment hosts.
- Change records: Log each feed or content import as a configuration change so that CM-related evidence shows who approved it and when it took effect.
- Wazuh coverage: Confirm that agents on hosts with restricted egress can still reach the Wazuh server on the internal network, and that the server retains logs for the period the organization's retention schedule requires.
- Time sources: Point all tools at internal time servers so that log timestamps and scan dates line up across systems.
Confirm the mirror and staging process with the vendor documentation for each tool before you build it, especially how feed signatures or content checksums are verified.
An air-gapped network has no direct path to any update source, so every feed, content release, package and signature must cross the boundary by approved media or a controlled transfer process. The main risk is not the tooling itself but stale content: a scanner with an old feed or an OpenSCAP profile from an outdated release will produce results that look valid but test against the wrong baseline.
- Transfer process: Define a documented import procedure with a named approver, a checksum or signature check, and a record of each transfer that enters the enclave.
- Feed age: Track the age of the Greenbone feed and the ComplianceAsCode content in every scan report, so that an assessor can see whether results used current content.
- Evidence export: Plan how Wazuh alerts, OpenSCAP reports and scan outputs leave the enclave for the assessor, and whether the assessor will review them on site.
- Offline secrets: Confirm how OpenBao unseals and backs up in a network with no external services, and who holds the recovery material.
- Identity: Keycloak must run locally with its own user store, since federation to external identity providers may not be available.
Before committing, ask each vendor and your own security team to walk through the offline update path step by step in a test enclave, and record the results as part of your evidence.
Which tool to deploy first, and when each fits
This table compares the five tools by the SP 800-53 control family each most often evidences, its license, a typical first use, and the gap that still needs a manual process. Read each row as a starting point: the organization still owns the control, and an independent assessor decides whether the evidence is sufficient.
| Tool | Control families it most often evidences | License and typical first use | Main gap needing a manual process |
|---|---|---|---|
| OpenSCAP | CM (secure configuration baselines), CA (assessment results) | LGPL-2.1-or-later. First use: scan a sample of hosts against an SCAP Security Guide profile and keep the XCCDF or ARF reports. | Scan results show pass or fail per rule only. Deciding which profile fits the system, documenting deviations and approving them still need a manual process. |
| ComplianceAsCode content | CM (baselines and settings as code), SI (repeatable hardening) | Content is open source, with the project's own license terms to confirm. First use: generate SCAP data streams, Ansible or Bash remediation for a RHEL, Ubuntu or similar host. | Content covers only the rules it ships for each platform. Mapping rules to the organization's chosen baseline and tailoring them is a manual step. |
| Wazuh | AU (log collection and review), SI (monitoring and file integrity), CM (configuration assessment) | GPL-2.0-only. First use: deploy agents on monitored endpoints, forward logs to the server and enable file integrity monitoring on critical paths. | Alerts need someone to triage and document the response. Log retention periods, review cadence and who reviews are policy decisions the organization must record. |
| Greenbone Community Edition (OpenVAS) | RA (vulnerability scanning), SI (flaw remediation tracking) | AGPL-3.0-or-later. First use: schedule authenticated or unauthenticated scans of in-scope networks against the Community Feed. | Scan output is not a risk decision. Triage, remediation timelines, risk acceptance and verifying fixes need a documented manual process. |
| Keycloak | IA (MFA and authenticator management), AC (account and session handling) | Apache-2.0. First use: connect applications to Keycloak as the identity provider and configure one-time password policies and passkeys. | Keycloak does not decide who gets access. Joiner, mover and leaver approvals, access reviews and identity proofing still run through HR and manual steps. |
| OpenBao | SC (key and certificate management), IA (secrets and credential lifecycle) | MPL-2.0. First use: store application secrets centrally and issue dynamic, short-lived database credentials with leases. | Key ceremonies, custody of root material and rotation policy require documented procedures. Confirm supported features for the required key types with the vendor. |
How the tools go from deployment to assessment-ready evidence
The sequence below turns a self-hosted stack into SP 800-53 evidence that an independent assessor can sample. Each phase produces a named artifact that feeds the next, so the final review works from records rather than recollection.
Scope the system and select the baseline
The organization defines the system boundary, its impact level and the SP 800-53B baseline it must meet, with the privacy baseline added where personal data is processed. DeployOpen can prepare the inventory of hosts, containers, identity services and data stores that the scope covers, but the organization owns the categorization decision and the resulting control selection. The step is done when the boundary and selected controls are written down and approved by the system owner, giving every later tool a clear target.
Deploy identity and secrets services
Keycloak is deployed inside the boundary to handle user and administrator authentication, including MFA with TOTP, passkeys and recovery codes under the one-time password and WebAuthn policies the organization sets. OpenBao is deployed to hold secrets and issue short-lived dynamic credentials with leases. The output is a configuration export showing MFA enforcement and a secrets inventory, which together support the identification, authentication and key management evidence. Confirm each feature and version against the vendor documentation for your environment before you rely on it.
Enable audit logging and integrity monitoring
Wazuh agents are installed on monitored endpoints and forward logs to the Wazuh server, which is configured to retain records for the period the organization's audit policy requires. File integrity monitoring is set up against a baseline of checksums and attributes for the directories and files in scope, with real-time or scheduled checks. The output is a log source list, a retention setting and a sample alert showing who changed a monitored file and when, which assessors often request to test audit and integrity controls.
Apply configuration baselines with OpenSCAP
The team selects a SCAP profile from ComplianceAsCode content that matches the operating system and the approved configuration baseline, then runs the oscap scanner against each host. Failed rules are either remediated, documented as approved deviations, or handed to the system owner for a risk decision. The output is a set of ARF or XCCDF result files and a list of accepted deviations, which provide the configuration baseline evidence an assessor will compare against the organization's documented settings.
Run vulnerability scans with Greenbone
Greenbone Community Edition is deployed to scan in-scope systems on a schedule the organization defines, using the Community Feed that updates daily. Scan results are reviewed by the security team, and findings are triaged into remediation tickets or documented risk acceptances. The output is dated scan reports, a remediation log and a record of how each high-severity finding was closed. The scanner's configuration and its feed update history should also be retained, since assessors may ask how the scan coverage was determined.
Collect and tag evidence to control IDs
Outputs from the earlier phases are gathered into a single evidence repository and tagged with the SP 800-53 control identifiers they support, for example mapping an OpenSCAP result to the configuration management control or a Wazuh alert to the audit review control. The tagging can be done in a spreadsheet or in OSCAL documents, and the organization decides which convention its assessors accept. The output is an index that links each control to its evidence, the date it was produced and the person responsible for it.
Review gaps and prepare for assessment
The control owner and the independent assessor's expectations are compared against the evidence index to find controls with missing, stale or failing evidence. Gaps are recorded as plan of action and milestones items with owners and dates set by the organization. DeployOpen can help prepare the review, but it does not issue assessment results. The step is complete when every selected control either has current evidence or an open, approved entry in the plan of action and milestones, which is what an assessor will examine first.
Controls the tools do not cover on their own
Scanners, log platforms and secrets managers produce machine evidence, but many SP 800-53 requirements are governance records that someone must write, own and review. The cards below cover the artifacts that remain manual and the decisions an assessor will expect a named person to have made.
Policies and procedures
Policy documents set the rules that tools then enforce, such as who may approve access, how changes are reviewed and how often scans run. Each policy needs a named owner, an approval date and a review cycle. An assessor will check that the configuration in OpenSCAP, Wazuh or Keycloak matches the written policy, so the two should be reconciled at each review. Confirm who signs off and where the approved versions are stored.
Roles and responsibilities
Tools record what happened, but they do not decide who is accountable for a control. A responsibility matrix should name the person or team that owns each control, the people who operate the tool, and the backup owner during absence. Without it, a Wazuh alert or an OpenBao lease expiry has no clear recipient. Ask for the matrix to be part of the system security and privacy plan.
Training records
Awareness and role-based training are shown through attendance records, completed course content and the training materials themselves. Automated tools cannot judge whether an administrator understands secrets handling or incident escalation. Keep dated records linked to each person's role, and confirm that privileged staff completed role-specific training before they receive production access.
Risk acceptance decisions
Scanner output lists findings, but a person must decide whether to fix, mitigate or accept each one. A risk acceptance records the finding, the compensating measure, the approver and the review date. Greenbone results without these decisions show open issues with no documented reasoning. Confirm the approval authority and the expiry rule before accepting any high-severity finding.
Plans of action and milestones
A plan of action and milestones lists each weakness, the planned remediation, the responsible owner and the target date. Tools can detect the weakness, but the POA&M is a managed record that someone updates when work is scheduled or delayed. Track it in a controlled repository, and confirm that status changes trace back to evidence such as a rescan result or a change ticket.
Personnel security screening
Screening, termination steps, transfer reviews and signed access agreements depend on HR and legal processes outside the platform. Keycloak can disable an account on request, but it cannot confirm that a background check was completed or that a departing employee returned equipment. Ask HR to keep signed agreements and termination checklists, and reconcile them against active accounts in the identity provider at a fixed interval.
Questions buyers ask about open-source tools for SP 800-53
Do assessors accept open-source tools as evidence for SP 800-53 controls?
Assessors accept evidence based on what it shows, not on the license of the tool that produced it. NIST does not endorse or certify any compliance tool, so the question is whether the output lets an assessor run the SP 800-53A procedure for a control: examine the artifact, interview the owner, and test the behavior. OpenSCAP reports, Wazuh alerts and file integrity records, and Greenbone scan results can each serve that purpose when the configuration, scope and timestamps are documented. Confirm with your assessor or authorizing official which tool outputs they will accept and how they want them retained.
How much effort does an open-source SP 800-53 tooling deployment take?
Effort depends on the number of systems in scope, how many control families the tools must cover, the existing logging and identity setup, and how much content must be tailored. NIST does not estimate implementation effort or timelines, so no figure applies here. The main drivers are the baseline selected under SP 800-53B, the number of distinct operating systems and environments, whether an identity provider already exists, and how much evidence must be mapped to control IDs. Ask any provider for a written scope that lists systems, tools, integrations and deliverables before agreeing to a price or schedule.
Can our team run open-source SP 800-53 tools without outside help?
Many teams can run the tools internally if they already have Linux administration, logging and identity skills and someone who owns the control documentation. The work that tends to need extra hands is first-time configuration, tailoring SCAP content to your baseline, wiring Wazuh, Keycloak and OpenBao into your environment, and organizing evidence for an assessment. Self-hosting also means your team owns patching, upgrades, backups and monitoring of the tools themselves. Decide which of these your staff can carry on a recurring basis before deciding how much to outsource.
What does the work require from our internal staff?
Internal staff keep decisions that only the organization can make. These include selecting the impact level and baseline, approving the system security and privacy plan, accepting or rejecting risk, and assigning a named owner for each control. Engineers provide access to systems, approve changes to the tools, and maintain the environments the tools run in. Compliance staff keep the plan of action and milestones and the assessment records current. Ask a provider to name which of these responsibilities it will perform and which it expects your team to hold.
How do tool outputs map to SP 800-53 control IDs?
The mapping is written by your organization, not produced automatically by any tool. OpenSCAP and ComplianceAsCode rules reference configuration checks that you can trace to CM-6 and related settings, Wazuh file integrity monitoring supports evidence for integrity and change monitoring controls, Greenbone results support vulnerability scanning under RA-5, and OpenBao and Keycloak output supports identity and key controls. Record each mapping in your system security and privacy plan with the tool version, configuration and retention location. Confirm each mapping with your assessor, and check the current control text in the NIST OSCAL catalog.
Which version of SP 800-53 should we target?
Target the release your authorizing body or contract names, because that determines the control text and assessment procedures you must satisfy. NIST's current release is Release 5.2.0 of 27 August 2025, which added SA-15(13), SA-24 and SI-2(7) and revised SI-7(12), with updates to several other controls. Use the OSCAL content from NIST's repository for your mapping so control IDs and text match the release. Confirm the required release with your agency, customer or assessor before building your evidence set.
What does a partner do and what does it not do?
DeployOpen plans, implements, customizes, hardens, documents and supports self-hosted open-source platforms in your own cloud accounts, data centers or air-gapped networks, and prepares the systems and evidence for an assessment. DeployOpen does not own your controls, does not issue audit reports, attestations or certifications, and does not decide whether a system meets SP 800-53. Only an independent assessor or the authorizing official can do that. Engagements are usually fixed-scope, with scope and deliverables agreed in writing before work starts, and ongoing support is available as a retainer.
What an implementation partner does and what it never issues
The organization owns its SP 800-53 controls, its risk decisions and its authorization. An implementation partner works on the systems and the evidence around those decisions: it plans the deployment of tools such as OpenSCAP, Wazuh, Greenbone, Keycloak and OpenBao inside your environment, configures them to the control settings your organization has selected, hardens them, writes the operating procedures, and assembles the records an assessor will ask to see. Those records include scan configurations and reports, file integrity baselines, authentication policies, secret lease settings and change history. The partner does not decide whether a control is satisfied, does not accept residual risk, and does not sign the system security and privacy plan on the organization's behalf.
Only an independent party issues the outcome. Depending on the program, that is an external assessor, an Inspector General, a FedRAMP-recognized third-party assessment organization or an authorizing official within your agency or company. Confirm with your assessor or authorizing official which artifacts they will accept and in what format before the tooling is built, because a scan report that is accepted in one program may not be sufficient in another. DeployOpen prepares systems, controls and evidence for these reviews. It does not issue assessment reports, attestations, authorizations or certifications, and it does not represent itself as an assessor.
DeployOpen works under a written, fixed scope. Before work starts, the scope, deliverables, responsibilities and timelines are agreed in writing, and a quote follows from that scope rather than from a published price list. Ask any partner to show which controls and systems are in scope, which artifacts it will hand over, who keeps the configuration and where the evidence is stored, and which work is excluded. Ongoing support, upgrades and continuous testing are priced separately as retainers, so confirm which recurring tasks are covered after the initial delivery.
