Kubernetes CIS benchmark compliance

Before any scan result means anything, you need a written decision on which CIS Kubernetes Benchmark recommendations apply to your cluster. Map each recommendation to the component that implements it, such as the kube-apiserver flags, kubelet configuration, etcd settings, or the policies you run in the cluster, and record who controls that component. Recommendations that a managed provider controls should be marked as inherited or out of scope, with the provider's documentation cited as the reason. Every exclusion needs a recorded reason and an owner who can defend it to an assessor. A recommendation you cannot test is not automatically a pass, so the record should say whether the control is inherited, excluded, or still owed by your team. The control plane itself often decides how much applicability work you face. If you run upstream kubeadm, you can set and verify most API server, controller manager, scheduler and etcd flags, and the evidence is the configuration files and the running process arguments. Some managed offerings do not let you change the control plane at all. In that case the provider's shared responsibility documentation becomes primary evidence, and you need to collect what the provider states about the controls it operates, plus what you can still show directly, such as node configuration, namespace policies and RBAC. Ask the provider which benchmark version and which profile its documentation maps to, because a mapping written for one version may not match the version you scan against. An air-gapped cluster or a custom kubeadm install can also change which recommendations apply in practice. Some checks assume internet-reachable image registries, package repositories or certificate authorities, and an offline design may legitimately differ from the default. Where a recommendation conflicts with your architecture, write a deviation rationale that states the requirement, the alternative control you use, the residual risk, and who accepted it. Do not treat a deviation as a silent exception. Reviewers will look for the reasoning, the compensating control, and evidence that the compensating control actually runs. Before you finalize the mapping, confirm three things in writing. First, the exact CIS Kubernetes Benchmark version and profile you are assessing against, since recommendations change between versions. Second, the current provider documentation for each managed component, because provider behavior changes over time. Third, the license terms for the benchmark itself: CIS describes its Kubernetes Benchmark PDF as freely available for non-commercial use, so if your organization or a partner uses it in commercial work, check with CIS whether a separate license applies before you build the mapping, reports or customer deliverables on it.

The safest way to close a benchmark finding is to treat each change as a control plane release, with an approval, a test and a way back. A wrong flag on kube-apiserver can stop every API call in the cluster, including the calls needed to roll the change back, so the plan should be settled before anyone edits a manifest. Group findings by component: kubelet settings, kube-apiserver flags, Secrets configuration, namespace labels, RBAC and etcd. Each group has a different blast radius and rollback path. For every change, record the current value, the target value, the nodes or namespaces affected, the approver and the test that proves the finding is closed. Findings that can only be closed by a risky change should be reviewed for a compensating control before they reach the change calendar.

Most remediation work falls into five patterns. The first is kubelet and kube-apiserver flags that the benchmark expects to be set explicitly, such as authentication and audit logging settings. Changes to these flags usually need a component restart, so schedule them node by node. The second is encryption at rest. Secrets are stored unencrypted in etcd by default, so kube-apiserver needs an encryption provider configuration, with keys held in a key management service you control. Existing Secrets stay in their old form until they are rewritten, so plan and verify that rewrite. The third is Pod Security Admission labels on each namespace. Apply Baseline or Restricted first in warn and audit modes so workloads that would fail show up before enforcement. The fourth is least-privilege RBAC that removes list and watch on Secrets from roles that do not need them. The fifth is etcd client and peer TLS, which protects traffic between members and from the API server.

Changes to control plane components should follow the version skew policy even when the Kubernetes version does not change. The policy requires kube-apiserver to be upgraded before kube-controller-manager, kube-scheduler and kubelets, and it allows kubelets to be up to three minor versions older than kube-apiserver. These rules matter for remediation because a flag that is valid on one minor version may be removed or renamed on another, so check each flag against the version you run before editing. In a high availability cluster, change one control plane node at a time, confirm the API server, etcd members and scheduler are healthy, then move on. For kubelet changes, drain the node, apply the change, restart the service, uncordon the node and confirm it reports ready before continuing. Take an etcd snapshot with etcdctl snapshot save before the first change, and rehearse the restore path with etcdutl in a staging cluster.

Decide the rollback before the change, not during the incident. For static pod manifests on kubeadm-built control plane nodes, keep a dated copy of the original manifest on the node and in the change record. Define the health signals that trigger reversion, such as the API server failing to respond or pods failing to schedule. Secrets encryption needs particular care: removing the encryption provider configuration does not decrypt data already written, so a rollback may require rewriting Secrets again under the restored settings. Restore from an etcd snapshot only when cluster state is damaged, because it discards every change made since the snapshot. Some findings are better closed by a documented compensating control, such as restricting network paths to the API server or enforcing a policy at admission, especially when the direct change would break workloads or cannot be applied on a managed service. Record the owner, the residual risk, the review date and the evidence that the control works.

A CIS finding is closed only when a repeated scan confirms it and behavioral tests show the cluster still works. Rerun the same benchmark version your scope cites, with the same scanner version, profile and cluster configuration that produced the original results. Save the raw output and diff it against the baseline by check ID. A check that moved from fail to pass needs a matching change record. A new fail is a regression to investigate, not noise to ignore. If the scanner version or benchmark version changed between runs, the diff does not prove anything, so rerun the baseline first. Scan results prove that a setting exists, not that the platform behaves correctly. Schedule a test workload set in a namespace that matches production settings and confirm the pods reach Running. Create a privileged test pod in a namespace labeled to enforce the Restricted Pod Security Standard, and confirm that the API server rejects it with an admission error. For encryption at rest, create a test Secret, then read its key under /registry/secrets in etcd with etcdctl get from a control plane node. The stored value should not be readable plaintext and should carry the prefix your encryption provider writes. Confirm that prefix in the Kubernetes encryption at rest documentation for your provider. For RBAC, impersonate a service account that lacks Secret rights and run kubectl auth can-i list secrets in each namespace. The answer should be no everywhere it is meant to be. Some failure modes pass the scan but still hurt the business. A hardening change can satisfy a check and then block the next minor upgrade, for example when a tightened admission policy or a kubelet flag conflicts with a controller or add-on that the newer release expects. Test the upgrade path in staging before accepting the change, and check the version skew rules for every node role. A second risk is a scanner that cannot see managed control plane settings. Managed services are covered by the benchmark, but a scanner running inside the cluster often cannot read API server or scheduler flags. Mark those checks as not verified, collect the provider's own configuration evidence, and do not count them as passes. The third risk is a finding closed by an undocumented exception, such as a scanner exclusion or a namespace label that silences a check. Every exclusion needs an owner, a reason, a compensating control, an approver and a review date. Decide the response to a failing check before the change window opens. Roll back when the change broke workloads, a security control or control plane health, and restore the previous manifests or configuration, then rerun the full test set. Record a deviation entry when the team decides not to apply a recommendation, and keep the reason, the compensating control and the expiry with the scan evidence. Retest when the result is ambiguous, the fix was applied but not proven, or the scanner could not reach the setting. Put these three triggers in the change plan with a named owner for each, so the engineer on call does not decide under pressure. Ask your platform team to show you which trigger each past failure produced and where that evidence is kept.