Kubernetes on OpenStack

Each Kubernetes feature that reaches into infrastructure depends on one OpenStack service, and that mapping decides which credentials, quotas and failure domains you must plan before the cluster is built. The cloud controller manager from cloud-provider-openstack registers Kubernetes nodes with Nova instances, reads node metadata such as addresses and zones, and provisions load balancers when a cluster asks for them. Storage, identity and secret encryption are handled by separate services, so a gap in any one of them shows up as a different kind of failure. Start by writing down, for each feature, which OpenStack service it calls and which project and account it uses.

The Container Storage Interface drivers handle persistent storage. The Cinder CSI driver provisions PersistentVolumes from Cinder volumes, which suits block storage for single-attach workloads such as databases. The Manila CSI driver provides shares for workloads that need shared file access. Many operators back Cinder with Ceph, and the Ceph documentation on block devices explains that Ceph can supply Glance images, Cinder volumes and Nova guest disks from one cluster. Confirm with your storage team which Cinder backend, pool and replication settings sit behind each storage class, because those settings govern performance and recovery more than the Kubernetes manifest does.

Octavia serves Services of type LoadBalancer, so every exposed Service may create a load balancer, a listener and a pool in your project. Confirm which Octavia provider and flavors your cloud offers, and set quotas for load balancers and floating IP addresses before teams start creating Services. Keystone authenticates every call the cloud provider makes. Use a dedicated service identity with an application credential scoped to the cluster's project, grant it only the roles the controller manager, CSI drivers and Octavia integration need, and plan how that credential is rotated. Keystone federation governs human access and is a separate decision.

Barbican is the OpenStack key manager and can supply keys for encrypting Kubernetes Secrets at rest. This matters because Kubernetes stores Secrets unencrypted in etcd by default, so encryption must be configured on purpose. Confirm with your OpenStack release and your Kubernetes distribution whether the KMS integration you plan to use supports Barbican, and which key lifecycle controls apply. Decide who can create and rotate the key, where its backup is held, and how you will restore etcd if the key is unavailable. Test a restore, because encrypted etcd snapshots are only useful if the matching key can still be retrieved.

The first decision is where etcd runs. The kubeadm high availability guide describes a stacked topology, where etcd members share nodes with the control plane, and an external topology, where etcd runs on its own cluster. Stacked control planes need fewer machines and fewer handoffs, which suits one team that owns both layers. External etcd suits larger estates or cases where a separate team operates the data store and its backups. Either way, etcd needs an odd number of members, and Kubernetes documentation recommends five members in production. Confirm with your platform team which OpenStack instances, placement rules and volume types the five members will use, because an etcd member on a slow Cinder volume can degrade the whole control plane.

Storage decisions shape the CSI configuration more than any other single setting. Storage classes in the cluster reference Cinder volume types, so each volume type you expose becomes a choice that developers will make, usually without reading the backend notes. List the volume types before the build, record which backend and performance characteristics sit behind each one, and retire any type you do not intend to support. If your OpenStack deployment uses Ceph, Ceph block devices can back Cinder volumes, so pool design and failure domains sit underneath every volume type. Manila is needed only when pods must share a filesystem across replicas. Ask application owners which workloads write to a common path from several pods before you deploy Manila, and plan its backup and capacity separately.

Tenancy comes next. A single OpenStack project for the cluster is simpler to bootstrap and gives one set of quotas, but every cluster, load balancer and volume then draws from one pool of capacity and one set of role assignments. Multi-project designs separate production from non-production and make ownership clearer, but they make quota design explicit. Each project needs compute, network, volume, load balancer and security group quotas sized for autoscaling and for the extra nodes a rolling upgrade temporarily requires. Octavia load balancers and Cinder volumes consume quota as Kubernetes creates Services and PersistentVolumeClaims, so a quota that fits day one may block the next upgrade. Ask your OpenStack administrators to confirm quotas in writing and to name who approves increases.

The upgrade plan has to track two release trains. Kubernetes releases about three times a year on a cycle of roughly 14 weeks, and the project maintains the three most recent minor releases. The version skew policy requires kube-apiserver to move one minor version at a time, ahead of kube-controller-manager, kube-scheduler and kubelets, and allows kubelets to be up to three minor versions older than kube-apiserver. That gives you room to stage node upgrades, but it is not a reason to fall behind. OpenStack supports upgrades between SLURP releases as well as between adjacent releases. The 2026.1 Gazpacho and 2025.1 Epoxy releases are SLURP releases, and 2027.1 Indri is also SLURP and in development. Map each Kubernetes minor version you plan to run against the OpenStack release beneath it, and confirm the cloud provider and CSI versions that release train supports.

An end-to-end test plan for Kubernetes on OpenStack should exercise every point where the cluster calls OpenStack, and each test needs a stated expected result before it runs. Run the plan in a test project that mirrors production networks, flavors, security groups and the cloud provider configuration, so that results carry over. Start with node lifecycle. Replace one worker VM through the Nova API, then confirm that the cloud controller manager registers the new Node object, applies the expected labels and removes the stale Node after the old instance is deleted. Repeat the check for a control plane node if your topology allows it, since losing etcd membership is a different failure from losing a worker.

Storage tests come next. Create a PersistentVolumeClaim backed by the Cinder CSI driver, mount it in a pod, and drain the node that holds the pod. Confirm that the volume detaches from the old instance in OpenStack, reattaches to the replacement node and remains readable with the same data. If you use Manila for shared file systems, mount the same share from two pods scheduled on different nodes, write from one and read from the other, and verify the access rules and mount options match what your storage team approved. Record any delay between drain and reattachment, because it becomes the basis for your recovery expectations.

Service tests cover Octavia. Create a Service of type LoadBalancer, confirm that the load balancer, listeners, pools and health monitor appear in OpenStack, and send traffic through the external address. Then delete the Service and check that every related Octavia object and any floating IP is removed, because orphaned load balancers keep consuming quota and can expose stale endpoints. For credential rotation, rotate the Keystone credential that the cloud provider uses, update the cloud configuration secret, and confirm that existing workloads keep serving traffic while new volume and load balancer operations succeed. Confirm the exact rotation steps with the release documentation for your provider version before you schedule the test.

Two tests protect recovery. Take an etcd snapshot with etcdctl snapshot save, restore it into an isolated environment with etcdutl, and verify that workloads, Services and Secrets are present. Secrets are readable only if the encryption key is still available, so confirm that the Barbican-backed KMS plugin can reach its key in the restore environment. For the upgrade rehearsal, check the version skew policy and the cloud provider's compatibility notes for the target Kubernetes minor version. Upgrade kube-apiserver first, then the controller manager, scheduler and kubelets in the order the policy requires, and rerun the node, storage and load balancer tests afterward.

Write acceptance criteria into the scope before any work starts, including the expected result, the owner and the pass or fail rule for each test. Keep the evidence for every run: command output, controller logs, OpenStack API records, configuration versions and timestamps. This evidence supports compliance readiness work, since assessors will ask how access, encryption and recovery were verified. DeployOpen prepares these systems and records, but independent auditors and certification bodies decide whether controls meet a framework's requirements.