Kubernetes for SaaS companies

The first decision in tenant isolation is how many clusters you will run and how tenants are placed inside them, because every isolation control in Kubernetes is scoped to a cluster. Within a shared cluster, the common building blocks are a namespace per tenant or per pricing tier, role-based access control (RBAC) bindings limited to those namespaces, Pod Security Admission levels set per namespace, and network policies that restrict traffic between namespaces. Each layer addresses a different failure. Namespaces give you a boundary for names, quotas and policy objects. RBAC limits which identities can read or change objects in that boundary. Pod Security Admission controls what a workload may ask the node for, such as host namespaces, privileged containers or host paths. Network policies control which pods can open connections to each other. Namespace separation on its own is one layer. It does not stop a container escape, a privileged pod in another namespace or traffic that a network policy does not block.

Pod Security Admission enforces the three Pod Security Standards (Privileged, Baseline and Restricted) per namespace through labels. A practical starting point is to set the Restricted level on tenant namespaces and to keep system namespaces on a level that the platform components actually need, then run the cluster in audit and warn modes before enforcement. Confirm for your chosen Kubernetes version and distribution how the admission labels are set and which workloads, such as monitoring agents or storage drivers, need an exception. Pair this with RBAC rules that deny tenant identities any access to cluster-scoped objects, to other namespaces and to Secrets listing, since Secrets are stored unencrypted in etcd by default unless encryption at rest is configured.

Shared or dedicated clusters for higher-value tenants is a design decision, not a default. Namespace isolation in one cluster shares the control plane, the node pool, the etcd data and the cluster-wide admission and CRD configuration. Some customers will accept that model under their contract and risk assessment. Others will require a dedicated cluster, a dedicated node pool or a separate account or data center. Make this choice with your risk owners and the contract terms in view, and record the reasoning, the tenants affected and the exit path. A dedicated cluster for one tenant adds an operating cost for each cluster, including upgrades, backups and monitoring, so ask for that cost to be shown separately in any quote.

Kubernetes does not store normal users as API objects. It has no user database and no resource that represents a person. Instead, the API server accepts identities from X.509 client certificates, bearer tokens, webhook token authentication or an authenticating proxy, and then applies RBAC to the identity it extracts. For end users and operators, the usual approach is OpenID Connect (OIDC), where the API server is configured to trust an identity provider and reads the username and group claims from the token. Decide which claims map to Kubernetes users and groups, then bind RBAC roles to those groups rather than to individual people. Confirm the OIDC settings supported by your distribution and identity provider, and test that revoking a user at the identity provider removes cluster access within the token lifetime you accept.

Service accounts used by tenant workloads need the same discipline. Disable automatic token mounting where a pod does not call the API, give each workload its own service account with a narrow role, and audit any ClusterRoleBinding that grants access across namespaces. Before go-live, verify the design with a test run: attempt cross-namespace reads, a privileged pod in a tenant namespace and a connection between two tenant namespaces, and record the results. Those test results and the RBAC and admission configuration form the evidence your team will need later for access control reviews.

A platform team keeps tenant deployments reproducible by making Git the only source of desired state. Namespaces, Deployments, ResourceQuota objects, network policies and Argo CD Application definitions all live in version-controlled repositories, and Argo CD continuously compares that desired state with the live cluster. When someone changes a workload by hand, Argo CD flags the drift and can revert it to what Git declares. The design decision to make early is repository layout: one repository per tenant, one monorepo with directories per tenant, or a shared base with per-tenant overlays. Argo CD AppProject boundaries should then limit which repositories, destination namespaces and cluster resources each tenant team can reach, so a tenant cannot deploy outside its own space. Promotion between environments should move the same tested manifests forward through pull requests or tags, with environment-specific values kept separate from the base configuration. Approvals belong in the merge process, and each promotion leaves an auditable commit history that also serves as change evidence. The GitOps controller itself needs the same discipline. Store the Argo CD installation configuration, its settings and its AppProject and Application resources in Git, and back up any state that is not reproducible from Git. Then rehearse a full rebuild of the controller in a separate cluster. Confirm the high availability topology and the recovery steps with the Argo CD release and the distribution you chose, because the exact procedure depends on those versions. Autoscaling works at two levels. The Horizontal Pod Autoscaler adjusts replica counts for a workload based on observed metrics, while a node-level autoscaler adds or removes worker nodes when pods cannot be scheduled. Both depend on resource requests. Requests tell the scheduler how much CPU and memory a pod needs, and autoscalers reason from them, so workloads without requests produce unreliable scaling. Limits cap what a container can consume. For noisy-neighbor control, combine per-namespace ResourceQuota and LimitRange objects, PriorityClass settings for critical tenant services and PodDisruptionBudgets for planned disruptions. Capacity planning then sets node pool sizes and headroom for scale-out, grouped by tenant tier. Confirm which autoscaling components your distribution supplies, which versions it supports and how it behaves during node drains and upgrades. Kubernetes releases arrive roughly three times per year on a cycle of about 14 weeks, and the project maintains only the three most recent minor releases. The version skew policy requires kube-apiserver to be upgraded one minor version at a time, before kube-controller-manager, kube-scheduler and kubelets, and kubelets may run up to three minor versions behind the API server. Delivery tooling has to fit inside that window. Argo CD, autoscalers, admission controllers and the ingress layer each have their own supported Kubernetes versions, so an upgrade can break reconciliation or scaling without any workload change. Plan the upgrade path as a repeatable process: test each step in a staging cluster, pin component versions in Git, and ask your distribution and each tool vendor for a compatibility statement for every target version before scheduling production work.

The cluster produces evidence for several SOC 2 criteria, but only if the company decides in advance which artifact answers which criterion and keeps that artifact in a retrievable form. For logical access in the CC6 series, the starting points are the Role and ClusterRole definitions, their RoleBinding and ClusterRoleBinding objects, and recurring access reviews that compare who holds those bindings against who should. For system operations in the CC7 series, the kube-apiserver audit policy and the exported audit logs show who did what to the control plane and workloads. For change management in the CC8 series, Git history and pull request approvals in the repository that feeds your GitOps controller show who proposed each change, who approved it, and when it reached the cluster.

Two other artifacts need specific attention. Secrets are stored unencrypted in etcd by default, so the encryption at rest configuration, the EncryptionConfiguration file passed to kube-apiserver, is evidence in its own right. Keep the file under version control, record when it changed, and confirm that existing Secrets were rewritten after encryption was enabled. For availability and recovery, etcd snapshot records from etcdctl snapshot save are only persuasive if you also keep restore test results made with etcdutl. A backup that has never been restored in a test gives an auditor little to rely on. Ask your auditor whether availability criteria are in scope, because that decides how much weight these records carry.

A Type 1 report describes the system and assesses whether controls are suitably designed as of a single date. A Type 2 report adds an opinion on whether those controls operated effectively over a period, and that period is what makes continuous collection necessary. An auditor selecting samples from a six or twelve month window will ask for proof that an access review happened in each cycle, that audit logs existed throughout, and that every change in the window has an approval trail. Evidence assembled in the final weeks cannot prove operation earlier in the period. Schedule exports of RBAC state, audit logs and pipeline approvals from the first day of the window, and confirm the retention periods cover the full period plus the time the auditor needs to test it.

Responsibility for the opinion sits with the parties named in the standards. The AICPA's 2018 description criteria, DC 200, govern how management describes the system, and management must write that description accurately, including which parts of the stack the cloud provider or distribution controls. Under AT-C section 205, the service auditor obtains written representations from management and forms the opinion. DeployOpen can plan, configure and document these controls and assemble the evidence, but it does not issue SOC 2 reports or attestations. Before the period begins, ask your service auditor which artifacts they will sample, in what format they want them, and how they will test the access review and change approval records.