SI-17 Fail-safe procedures: requirements and evidence

SI-17 fail-safe procedures: what the control asks for

A tailoring decision or a contract has put SI-17 in scope, and the team must choose it deliberately, because SP 800-53B does not assign SI-17 to any baseline. It applies only when the organization selects it through tailoring or when an overlay includes it. In plain terms, SI-17 requires the organization to define the failure conditions that matter for the system, define the fail-safe response to each one, and then carry those responses out when the conditions occur. A response can be an operator alert, a controlled shutdown, or a move to a defined safe state, such as stopping a process that controls physical equipment or holding transactions in a queue until the system recovers. The organization owns these definitions. An implementation partner can help write them and test them, but the decisions about what counts as a failure and what the safe outcome is belong to system owners and the people who understand the operational impact.

SI-17 is easy to confuse with three neighbors. SC-24 concerns preserving a known system state when failure occurs, so the question there is what state the system is left in. SI-13 concerns predicting failure, for example by tracking mean time to failure and staging spare components before they are needed, so it is about avoiding outages. The CP-12 and CP-13 related controls cover safe and secure shutdown and alternate security mechanisms as part of contingency planning, which overlaps with SI-17 but is not the same requirement. A useful first step is a short mapping table in the system security plan that shows which failure conditions each of these four controls covers, so an assessor does not find a gap between them or a double count.

Two things make SI-17 hard in practice. First, failure conditions are often never enumerated. Teams list hardware faults, power loss and network outages, but rarely software conditions such as a stalled queue, a failed authentication service, a corrupted configuration load or a logging pipeline that stops silently. Second, many fail-safe responses were written for physical equipment, while a modern platform is largely software-driven and its safe state is a set of application behaviors. Assessors test SI-17 by examining documentation of the fail-safe procedures, the system design and the configuration settings that implement them, so the evidence must show the defined conditions and responses in the actual system. Confirm with your assessor which artifacts they expect, and begin by listing failure conditions from the system architecture rather than from the hardware inventory.

How an assessor tests SI-17 fail-safe procedures

This table walks through the single SP 800-53A determination statement for SI-17 and each examine object the assessor reviews, with what they look for and where assessments usually stall. Read it as a checklist of evidence to prepare before the assessor arrives.

Assessment element	What the assessor checks	Common finding
Determination statement	Whether the organization has defined fail-safe procedures for the system and carries them out when the defined failure conditions occur	Procedures exist, but no test or log shows they were carried out when a failure condition occurred
Examine: fail-safe documentation	That the documentation names the failure conditions, the predefined response for each, and who acts on it, such as alerting operators or shutting down safely	Procedures are generic and name no specific failure conditions for this system
Examine: memory protection safeguards list	That the list of safeguards protecting system memory from unauthorized code execution is current and matches the fail-safe scope	Safeguard list is missing or was not updated after the system changed
Examine: system design documentation	That the design shows where failures are detected, how the system reaches a known safe state, and which components the procedures cover	Design documents describe normal operation only and omit failure paths
Examine: configuration settings and documentation	That configured thresholds, alert triggers and shutdown behavior match the written procedures	Settings differ from the documented procedure, or no owner reviews the configuration
Organization-defined parameters	That the three SI-17 parameters are filled in with concrete values, so the assessor can test against them	Parameters are left as placeholders, making any test result ambiguous
Interview and test methods	Staff explain how they respond to a failure, and the assessor may trigger a controlled failure to observe the response	Responses were never exercised, so staff describe steps no one has performed

How to implement SI-17 fail-safe procedures in sequence

This sequence takes SI-17 from a tailoring decision to a tested, maintained set of fail-safe responses, with each step producing a record an assessor can examine. The organization owns every decision and record, and an implementation partner or internal engineers prepare the systems and evidence.

Confirm SI-17 selection and record the decision
Start by confirming that SI-17 is in scope. SP 800-53B does not assign it to any baseline, so it enters a system only through tailoring or an overlay that the authorizing official accepts. Record the selection, the reason for it and the overlay or tailoring decision in the system security and privacy plan. Note that FedRAMP's Rev5 baselines do not require SI-17 at Classes B, C or D, so a FedRAMP system must justify its inclusion on its own terms. The output is a plan entry that names SI-17 and its rationale, approved by the system owner and the authorizing official.
Build the failure condition register
List the failure conditions that could affect each component in the system boundary, such as a lost data feed, a failed authentication service, a stalled queue or a control plane that cannot reach its safety logic. Use architecture diagrams and system design documentation as the starting point, since assessors examine those documents too. Then fill the three organization-defined parameters for SI-17 by stating which conditions count, which responses are required and the time or state thresholds that trigger them. The output is a signed condition register that maps each condition to its component, parameter values and owner.
Write a named response for each condition
For every condition in the register, write the fail-safe response. Responses range from alerting an operator and holding a transaction to a controlled shutdown that leaves data consistent. Assign one named owner per response who can execute it or authorize it, and state what the owner must do first. Where a response depends on another control, such as contingency planning under CP-12 or CP-13, or the known-state failure behavior in SC-24, cross-reference that control so the runbook does not contradict it. The output is a set of response runbooks, one per condition, each with an owner and a version date.
Configure detection and alerting for each condition
Set up monitoring that detects each condition and raises the response defined in its runbook. Tools such as Prometheus can collect metrics from instrumented jobs and store them as time series for querying, and Grafana Alerting can build alert rules from queries across data sources and route them to the owner. Test that each alert fires on the condition it is meant to detect and reaches a person who can act. The output is an alert configuration export and a map that links every register entry to a live rule. Confirm with each tool vendor that the feature set you need is supported in your version.
Test each response outside production
Run each fail-safe response in a non-production environment that matches the production configuration as closely as practical. Inject the failure condition, confirm that the alert fires, confirm that the owner executes the runbook, and confirm that the system reaches the defined safe state. Record the date, tester, inputs, observed behavior and any gap. Where a test fails, correct the runbook or configuration and retest. SP 800-53A lists examine and test methods for SI-17, so keep the test records in a form an assessor can review. The output is a test record for each condition with a pass, fail or remediated status.
Hold fail-safe responses under change control
Bring the register, runbooks and alert rules under the configuration change process. Any change to architecture, components, data flows or failure thresholds triggers a review of the affected conditions and responses, and the retest of changed responses. Set a review cycle for the register that the organization defines in its policy, and record each review even when nothing changes. The output is a change log that links each architecture change to the register entries it touched and the retest evidence that followed. Ask any outside partner to show how its change records feed into this log before work starts.

Evidence that passes an SI-17 assessment

SI-17 is assessed largely on documents and configuration, so the evidence has to show that fail-safe procedures were defined, built into the system and exercised. The six artifacts below connect those steps, and each one should be tied to change control so it stays accurate between assessments.

Fail-safe procedure document

This is the core artifact. It lists the failure conditions the organization has defined and, for each one, the predefined response, such as alerting operators, halting processing or shutting down components in a known safe order. An assessor checks that every condition traces to a defined parameter value in the control and that the responsible roles are named. Keep it under version control and require review whenever a new component or failure mode is added.

System design documentation

Design documents show where fail-safe behavior is built into the architecture, for example which services fail closed, where watchdogs sit and how a failed component hands off to its safe state. Assessors compare these diagrams with the procedure document to confirm the two describe the same system. Update the design record whenever a boundary, dependency or data path changes, and record the date and approver on each revision.

Configuration settings and change history

Configuration files and settings that implement fail-safe behavior, such as timeout values, health check thresholds and automatic shutdown triggers, are examined directly. Assessors look for the exported settings, their associated documentation and a change history that shows who altered a threshold and why. Store configuration in version control so every change has an author, a reason and a link to an approved change request.

Test records showing responses ran

A document that describes a fail-safe response is weaker than a record that shows it worked. Keep dated test results from controlled failure injections, showing the triggering condition, the observed operator alert and the resulting safe state. Assessors use test records to confirm the procedure runs as written. Repeat tests after significant changes and record any failed response along with the corrective action.

Alert rule exports and sample alerts

Exported alert rules show the failure conditions the monitoring stack watches for and the notification route each one triggers. Pair them with sample alert records from real or test events, which prove the alert reached a named responder. Grafana Alerting and Prometheus rule definitions can be exported and kept in version control. Confirm that each defined failure condition has a rule, and that the rule name matches the procedure document.

Review record and approvals

The review record shows that the fail-safe procedures, design, settings and tests were examined on a defined schedule and after relevant changes, with named reviewers and dated sign-offs. Assessors check that findings led to updates in the procedure document or configuration. Store reviews in the same wiki or document system as the procedures, so a rollback or comparison of any two versions is possible during an assessment.

What SI-17 fail-safe procedures cost to meet

This table breaks the effort behind SI-17 into cost components, showing who is paid, whether each cost is one-time or recurring, and what pushes it up or down. NIST does not estimate compliance costs, so use the rows to structure a written, scoped quote rather than a published figure.

Cost component	One-time or recurring	What drives it up or down
Engineering time to enumerate failure conditions and design responses	One-time, revisited on major change	Up with the number of components and failure modes; down when conditions are grouped and common responses are reused
Alerting tooling or licenses	Recurring	Up with commercial per-seat or per-source licensing; down with self-hosted open-source alerting such as Grafana Alerting and Prometheus, which still carries hosting and staff time
Testing time for fail-safe scenarios	One-time per scenario, then recurring on change	Up with safety or availability impact and the number of scenarios; down when tests are automated and rerun from a single suite
Evidence upkeep (design records, configurations, alert rule exports)	Recurring	Up with frequent configuration change and manual record keeping; down when design and alert definitions are version controlled
External review of the design and test results	Recurring at each assessment cycle	Up with system criticality, authorization scope and reviewer rates; confirm with the reviewer in writing
Auditor or assessor sampling time	Recurring at each assessment	Up with the number of documented conditions the assessor samples; down with a clear, indexed mapping from each defined condition to its response and evidence

Self-hosted open-source tools that produce SI-17 evidence

Many SI-17 procedures begin with an alert: a condition is detected, operators are told, and a defined response starts. Grafana Alerting (AGPL-3.0-only) builds alert rules from queries and expressions across multiple data sources and alerts on metrics and logs together, so one rule set can watch application telemetry, infrastructure signals and log patterns. Teams can create, manage and act on alerts from a single view, which helps when the assessor asks who received an alert and what they did next.

Rule definitions as evidence: Exported or version-controlled alert rules show which failure conditions the organization has defined and when each rule was changed.
Multi-source coverage: Rules that combine metrics and logs can reflect the defined failure conditions for a system, provided the data sources are connected and tested.
Single-view handling: Alert acknowledgment and action history in one place can support evidence that operators were notified and responded.
Notification routing: Confirm with your own configuration which contact points and channels are used and whether delivery is logged.
Testing the path: Test alerts against a controlled failure so the evidence shows the alert fired, reached a person and triggered the defined procedure.

Confirm the specific alert history retention, export options and audit features of your Grafana edition and version before relying on them as evidence.

Which parts of SI-17 are worth outsourcing

The split for SI-17 is simple to state: the organization must own the decisions about what counts as a failure and what the system does about it, while an implementation partner can build the machinery that detects, alerts and tests those responses. The failure condition register is the core of the control. It lists which failures matter, such as loss of a safety interlock, a data pipeline stalling or a controller losing its upstream feed, and what the predefined response is for each. Choosing between alerting an operator, degrading service or shutting down safely is a business and, in some sectors, a safety decision. Those choices belong to system owners, operations leadership and, where physical harm is possible, the people accountable for safety. A partner can document them, but should not decide them.

Much of the surrounding work is well suited to an implementation partner on a fixed-scope engagement. That includes configuring alert rules and routing in self-hosted tools such as Grafana Alerting, collecting the time series that show a failure condition was met (for example with Prometheus), building a repeatable fail-safe test harness that forces each defined condition in a non-production environment, packaging the evidence into a consistent folder structure, and running a readiness review before the assessor arrives. Scope these items in writing: name the systems, the failure conditions in scope, the test environments and the evidence formats. Ask whether the test harness will run against production-like configurations and who signs off on the results. Confirm the deliverables and acceptance criteria before any work begins.

Outsourcing preparation does not transfer ownership. An independent assessor still has to examine the documentation, system design and configuration settings, interview the people who run the fail-safe procedures, and test the behavior against the determination statement in SP 800-53A. Only that assessor's findings count toward an authorization or attestation. A partner's readiness review is an internal check that reduces surprises; it is not an assessment. Plan for the organization to keep its own evidence of who approved each failure response and when it was last exercised, because those records must still exist and be current when the assessor asks for them.

Who does what for SI-17: your team, a partner and the auditor

Your organization owns SI-17. The control requires defined fail-safe procedures that run when defined failure conditions occur, so the decisions about what counts as a failure and what the system must do in response belong to the people accountable for the system. Those decisions are organization-defined parameters, and an assessor will check that they exist, are approved and match how the system behaves.

Failure conditions: Define which failures trigger a fail-safe procedure, such as loss of a dependent service, sensor fault or integrity alarm, and record why each one matters to the system's mission.
Response procedures: Decide whether each condition should alert operators, degrade service, hold state or shut down safely, and document who acts and in what order.
Risk acceptance: Record which residual risks the business accepts when a fail-safe response reduces availability, and obtain sign-off from the authorizing official or equivalent owner.
Operator decisions: Assign operators who can make manual calls during an event and confirm they have the access and training to do so.
Ongoing ownership: Keep the procedures current when architecture changes and review them as part of the wider SI family governance.

Before any assessment, confirm that the failure conditions, parameters and sign-offs are written down in one place that the assessor can examine.

Where SI-17 implementations usually fall short

SI-17 asks for fail-safe procedures to run when defined failure conditions occur, so the gaps tend to sit between the written procedure and what happens at 2 a.m. The cards below cover the places where failure conditions, responses, people, documentation and evidence drift apart.

Failure conditions never enumerated

Many teams have a general statement that the system should fail safely but no list of the failure conditions that trigger it. SI-17 depends on defined conditions, because the organization must specify which failures the procedures cover. Check whether each condition, such as loss of a data source, a failed control plane, or a dependency timeout, is named, owned and tied to a specific fail-safe response. A list that cannot be traced to system components is a sign the definition is incomplete.

Responses written but never tested

A fail-safe procedure that has never been exercised is an assumption. Assessors look for evidence that the defined response actually runs when the condition is induced, not only a document describing it. Ask for test records showing the trigger, the observed system state and the recovery steps. Fault injection in a staging environment, with results kept, is a common way to produce this evidence.

Responses that need an absent operator

Some fail-safe procedures assume a person will notice an alarm and take a manual step, but nobody is on call outside business hours. When the operator is absent, the system may stay in an unsafe state for hours. Check that each response either completes automatically or has a named, reachable role with a tested escalation path. Confirm the response still works during holidays and staff turnover.

No link to the system security plan

Fail-safe behavior is often designed by engineering and never recorded in the system security and privacy plan. That leaves the authorizing official and assessors with no way to confirm the procedure is in scope or that its owner has accepted it. Map each defined fail-safe procedure to the SI-17 implementation statement, the affected components and the responsible role, and keep that mapping current when the design changes.

Alerts without a defined response

Monitoring tools can announce that a failure occurred, but an alert with no defined action is not a fail-safe procedure. SI-17 is about predefined responses, so each alert tied to a failure condition should name the action, the decision owner and the point at which the system shifts to its safe state. Open-source alerting such as Grafana Alerting can route these alerts, but the response logic still has to be written and tested.

Evidence gathered once, not refreshed

Fail-safe evidence captured at go-live becomes stale after patches, new components, network changes or new integrations. A design change can add a failure mode that no procedure covers. Set a trigger for re-review in change control, rerun the relevant fault tests after significant changes, and version the configuration and test records so an assessor can see what was true at each point in time.

Questions buyers ask about SI-17 fail-safe procedures

Is SI-17 fail-safe procedures required by default under NIST SP 800-53?

No. SP 800-53B does not assign SI-17 to the low, moderate, high or privacy baseline, so it applies only when your organization selects it through tailoring or an overlay. FedRAMP's Rev5 baselines do not require it at Class B, C or D either. Appendix C of SP 800-53 Rev. 5 marks it as implemented by the system and as an assurance-related control, which is why it often appears in an assessor's review of a system design. Check your authorizing body's requirements and any contract language, because a customer or regulator can still ask for it by name. Confirm the current FedRAMP text in the FedRAMP Rev5 control reference before you assume it is out of scope.

How long does it take to implement SI-17, and what drives the effort?

No supplied figure gives a duration, and NIST does not estimate compliance costs or timelines, so ask any partner or assessor for a written estimate tied to your scope. The effort depends mainly on how many failure conditions your system has, how many organization-defined parameters you must set, and how much of the failure response already exists in runbooks, monitoring and recovery procedures. Three parameters are defined for SI-17 in Release 5.2.0, so the decisions are bounded, but each one needs an owner and a justification. Systems with many dependencies, legacy components or unclear ownership take longer to document and test.

What does the internal team need to provide for SI-17?

The internal team owns the decisions, so it needs to supply the people and facts that define the control. Typical inputs include:

Named owners for each failure condition and the fail-safe action taken
Architecture and design documentation showing where the system can fail and how it behaves
Access to configuration settings and the runbooks used during past outages
Sign-off on the values chosen for the organization-defined parameters

Without these inputs, any partner can only draft procedures that nobody will operate. Plan for review time from system owners, because they must confirm that the fail-safe behavior matches how the system actually runs.

Can we implement SI-17 in-house without an outside partner?

Yes, if your team already understands the system design and has the security and operations skills to document and test failure responses. Organizations that do this themselves usually need a clear owner, a written procedure set, configuration evidence and a test record. Outside help is most useful when the system spans many components, when the team lacks time to write and test procedures, or when an assessor will need a clearly separated evidence package. An assessor will not accept your own description alone, so decide early who will gather and test the evidence, and whether that person can be independent of the design work.

How does SI-17 differ from SC-24 and SI-13?

SI-17 covers predefined responses to failure conditions, such as alerting operators or shutting down safely. SC-24 covers failure in known state, meaning the system preserves defined system state so that failures do not leave information exposed or inconsistent. SI-13 covers predictable failure prevention, such as tracking mean time to failure and preparing standby components so that failures are replaced before they cause outages. The Release 5.2.0 catalog lists CP-12, CP-13, SC-24 and SI-13 as related controls, so an assessor will expect your fail-safe procedures to fit with those controls rather than duplicate them. Document each control's scope separately so the evidence for one does not get counted as proof for another.

What evidence will an assessor ask for first under SI-17?

Assessors using the SP 800-53A procedure for SI-17 start with documentation. The examine objects include documentation addressing fail-safe procedures for the system, a list of security safeguards protecting system memory from unauthorized code execution, system design documentation, and system configuration settings with their associated documentation. Expect them to ask for the fail-safe procedures themselves, the design records that show the failure conditions, the configuration files that implement the responses, and a record of how the parameters were set. Keep these in version control with dated approvals so the assessor can see what changed and when.

What does a partner deliver under a fixed-scope SI-17 agreement?

DeployOpen plans, implements and documents systems inside your own infrastructure and prepares the controls and evidence for an assessment. Under a fixed-scope agreement, scope, deliverables and timelines are agreed in writing before work starts, so you should expect a written list of the failure conditions covered, the fail-safe procedures drafted, the configuration changes made, and the evidence package assembled. The organization still owns the control and approves the parameter values. DeployOpen does not issue audit reports, attestations or certifications, so an independent assessor must still test the control. Ask any partner to state in writing what is excluded, including assessment support and ongoing operation.

How DeployOpen scopes and prepares SI-17 evidence

DeployOpen approaches SI-17 Fail-safe procedures as a scoped engineering task inside your own infrastructure. We help define the failure conditions and fail-safe responses your organization selects, build and test them on self-hosted platforms, and document the evidence. Your organization owns the control and its assessment outcome, and an independent assessor reaches the conclusions.

How we scope

Confirm with the control owner whether SI-17 is selected in your baseline, tailoring or overlay, since SP 800-53B does not assign it to any baseline.
Agree in writing which systems, failure conditions and fail-safe responses fall within the engagement before work starts.
Record the organization-defined parameters for SI-17 so the failure conditions and required responses are explicit and testable.

How we build

Map each defined failure condition to a documented response, such as operator alerting or a safe shutdown sequence, on the self-hosted platform.
Configure alerting, such as Grafana alert rules fed by Prometheus metrics, so failure conditions reach operators through a defined route.
Test each fail-safe path in a controlled environment and record what happened, including whether the response matched the documented procedure.

How we hand over

Deliver the fail-safe procedure documentation, system design notes and configuration settings that the SP 800-53A examine methods would review.
Store procedures and change history in a wiki or document system such as XWiki, so every revision shows what changed, when and by whom.
Walk your operations and compliance staff through the test records, open items and the evidence an independent assessor will ask to see.

How we support

Ongoing support, upgrades and continuous testing are available as retainers when failure conditions or system design change.
Review fail-safe responses after platform upgrades, so documented procedures stay aligned with the running configuration.
Confirm with your assessor and control owner which evidence they need for SI-17 before each assessment cycle begins.

SI-17 Fail-safe procedures