Kubernetes is the base of a platform
Kubernetes schedules containers across a pool of nodes, restarts failed workloads and exposes a declarative API for deployments, services, storage claims and configuration. It is licensed under Apache-2.0. On its own, it does not choose a network plugin, storage driver, traffic entry point, registry, identity provider or backup method. Those choices make up most of the work of a self-hosted platform.
Start with the workloads. Stateless services, batch jobs, data stores and GPU jobs place different demands on node pools, storage and scheduling, and a shared platform that hosts regulated data needs stricter defaults than a single team's cluster.
Pick a distribution for its defaults and lifecycle
Each option runs conformant Kubernetes. They differ in hardening defaults, packaged components, air-gap support and upgrade tooling.
| Distribution | What it gives you | Consider it when |
|---|---|---|
| kubeadm | The upstream bootstrap tool. It builds a minimal cluster and leaves add-ons and hardening to you. Its client certificates expire after one year by default. | You want close alignment with upstream and will own every add-on and certificate renewal. |
| RKE2 | A Rancher distribution focused on security and compliance. Its defaults aim to pass the CIS Kubernetes Benchmark with minimal changes, it can enable FIPS 140-2 compliance, and it documents air-gapped installs. | Hardening, regulated environments or disconnected networks are early requirements. |
| K3s | A single binary under 100 MB that packages Flannel, CoreDNS, Traefik and a local-path provisioner. SQLite is the default datastore; etcd, MySQL and Postgres are supported. | Edge sites, small footprints or many small clusters matter most. |
| OKD | An opinionated distribution, closely related to Red Hat OpenShift, that preinstalls Operators and runs nodes on an immutable OS managed through cluster APIs. | You want an integrated platform and accept its choices for node OS and updates. |
Decide the platform layers up front
These components decide how workloads communicate, persist data and receive traffic. Changing them later usually means a migration.
Networking (CNI)
Choose a CNI plugin that enforces NetworkPolicy. Plan pod and service CIDRs so they do not overlap with data center ranges or VPNs, since changing them after installation is disruptive.
Storage (CSI)
Map storage classes to backends such as Ceph RBD or a SAN driver. Decide reclaim policy, expansion and snapshot support, then test a node loss with a stateful workload attached.
Traffic entry
Use Gateway API or a maintained Ingress controller, plus a load balancer implementation on bare metal. The Kubernetes project ended maintenance of the community Ingress NGINX controller in March 2026, so clusters still using it should plan a move.
Registry and supply chain
Run a private registry such as Harbor with vulnerability scanning, and allow pulls only from approved registries. Air-gapped clusters need every image, chart and OS package mirrored before installation day.
Size the control plane and node pools
Run three control plane nodes behind a load balancer or virtual IP. etcd needs an odd number of members, usually three or five, stacked on the control plane or on dedicated hosts. etcd is sensitive to disk write latency, so give it fast local storage.
Size nodes from resource requests, pod density, failure headroom and daemon overhead. Separate pools by workload class, such as general services, data stores or GPU jobs, using taints, labels and quotas. Keep enough spare capacity to drain a node for maintenance.
A disconnected cluster needs a local registry, mirrored charts, an internal OS package source, internal time and DNS, and a controlled path for updates. RKE2 and K3s document air-gapped installs.
Connect kubectl and the API to your identity provider
Kubernetes has no objects for normal user accounts. People authenticate through client certificates, OpenID Connect tokens or another configured method, and RBAC decides what they can do. The practical setup is usually OIDC: the API server trusts an issuer such as Keycloak or Microsoft Entra ID, and users sign in through a kubectl credential plugin that caches a short-lived token.
Map identity provider groups to Roles and ClusterRoles through bindings kept in Git, and reserve cluster-admin for a small group. Keep an offline break-glass credential for identity provider outages and test it on a schedule. Automation should use service accounts with short-lived tokens.
Harden against the CIS Kubernetes Benchmark
Use the CIS Kubernetes Benchmark as the baseline, record each deviation with a reason, and rerun the checks after every upgrade.
| Area | Control | How to check |
|---|---|---|
| API server | Limit anonymous access to health endpoints, enable audit logging and restrict network reach to the API. | An unauthenticated request to a resource is rejected and audit events reach central logs. |
| Workloads | Apply Pod Security Admission at baseline or restricted per namespace. | A privileged test pod is refused in a restricted namespace. |
| Network | Default-deny NetworkPolicy per namespace, then allow documented flows. | A pod cannot reach another namespace's database without a policy. |
| Secrets and etcd | Encrypt Secrets at rest, preferably through a KMS provider, and use TLS for etcd peers and clients. | Secret values in an etcd snapshot are not readable in plain text. |
| Nodes | Harden the host OS, keep kubelet authentication and authorization on, and patch on a schedule. | Benchmark results are stored with the change record. |
Back up cluster state and upgrade one minor version at a time
etcd holds the cluster state, including RBAC and Secrets. Take scheduled, encrypted snapshots and copy them off the cluster. A restore means stopping all API servers, restoring every etcd member and restarting the API servers, so rehearse it. Snapshots do not contain persistent volume data; back up stateful workloads with a tool such as Velero or application-native backups, and keep manifests in Git.
Upstream Kubernetes ships about three minor releases a year, maintains the three most recent, and gives each about one year of patches. As of October 2026, kubernetes.io lists 1.37, 1.36 and 1.35 as maintained. Distributions follow on their own schedules.
The API server must not skip minor versions. Upgrade kube-apiserver first, then the controller manager and scheduler, then drain and upgrade workers; kubelets may lag the API server by up to three minor versions but never lead it. Before each step, check charts for removed APIs, confirm add-on compatibility and take a fresh etcd snapshot.
Kubernetes operating ownership
A shared cluster has several owners. Write them down before the first production workload arrives.
Platform team
- Runs the control plane, node pools and add-ons.
- Owns etcd backups, restore tests and certificate renewal.
- Plans and executes minor version upgrades.
Application teams
- Own manifests, charts, resource requests and probes.
- Respond to alerts for their workloads.
- Back up data in their persistent volumes.
Security
- Approves RBAC roles, Pod Security levels and registry policy.
- Reviews benchmark results and exceptions.
- Holds the break-glass procedure and audit logs.
Change record
- Versions of the cluster, add-ons and node images.
- Upgrade runbook with the last tested restore date.
- Hardening exceptions with an owner and review date.
Kubernetes deployment questions
Should we run Kubernetes on bare metal or on virtual machines?
Both work. Virtual machines on Proxmox VE or OpenStack make node replacement easier. Bare metal removes a layer and can suit GPU or storage-heavy workloads, but hardware failure becomes part of the cluster runbook.
How do we move existing applications onto the cluster?
Start with a stateless service that already runs in a container, define it in Git and run it alongside the current host. Move stateful systems after storage, backup and restore have been tested with that workload.
What happens if we fall behind on upgrades?
Unmaintained versions stop receiving security patches, and catching up takes several sequential upgrades because the API server cannot skip a minor version.
Do you certify the cluster as compliant?
No. DeployOpen hardens clusters, collects evidence and helps teams prepare for SOC 2, ISO 27001, DORA or NIS2 assessments. Audit reports and certifications come from your auditor or certification body.
Scope and handover
Effort depends on the distribution, the number of clusters, air-gap requirements, storage backends, identity integration, workload migration and the hardening baseline. Add-ons and the upgrade process often take more time than the initial install.
DeployOpen plans, deploys and hardens the cluster, connects OIDC sign-in, sets up backups, runs upgrades and documents the result. Handover includes the cluster inventory, add-on versions, RBAC model, benchmark exceptions, restore evidence, the upgrade runbook and a support route.
