Kubernetes GPU scheduling with dynamic resource allocation

Dynamic Resource Allocation splits GPU scheduling into three objects owned by three different actors. A DRA driver publishes the devices it finds on each node as ResourceSlice objects, the platform team defines DeviceClass objects that describe which devices may be requested, and application teams create ResourceClaim or ResourceClaimTemplate objects that a pod references. The scheduler connects the three. Before you size a cluster or write a single manifest, confirm which DRA API version and feature state your Kubernetes release provides, and read the documentation for your specific driver, because driver behavior determines what appears in the slices.

A ResourceSlice is a cluster-scoped record of devices that a driver makes available, grouped into pools. For GPUs, each slice typically describes the devices on one node, along with attributes such as model and memory. Application teams never write these objects, and platform engineers should not edit them by hand, because the driver rewrites them when hardware or configuration changes. After installing the driver, check that every GPU node has a slice and that the device count and attributes match the hardware you bought. A missing or incomplete slice is the most common cause of workloads that cannot be placed, and it is cheaper to catch at this stage than during a production rollout.

A DeviceClass is a cluster-scoped object that the platform team writes to define a category of device, usually with CEL selectors on driver-published attributes and, where the driver supports it, configuration passed to the device. Claims refer to a DeviceClass by name, so the classes form the catalog of what teams may ask for. Decide how granular the catalog should be. One class per GPU model is easy to explain to application teams, while classes for partitioned or shared device profiles add choice and also add more ways to request the wrong thing. Keep the list short and document the intent of each class in the object's annotations or in your internal catalog.

A ResourceClaim is a namespaced request for one or more devices from a DeviceClass, and it can narrow the choice with its own selectors and count mode. A ResourceClaimTemplate generates a new claim for each pod that uses it, which suits pods that do not need to share a device. A single claim can instead serve several pods that must see the same allocation. Pods reference a claim through spec.resources.claims and the matching entry in the container's resources. Application teams own these objects, and the platform team should set namespace quotas and review which classes each namespace may use. The scheduler allocates devices when it places a pod, recording the result in the claim's status. If no slice contains a device that satisfies the class and the claim's selectors, the pod remains Pending and the scheduler reports the reason in the pod's events. Watch for Pending pods in GPU namespaces and alert on them, because a selector typo produces the same symptom as a full cluster.

The first decision in device sharing is which unit you share, because each unit creates a different isolation boundary. In Dynamic Resource Allocation, a ResourceClaim requests devices, and the containers or pods that reference that claim receive the allocated devices. You will typically evaluate three patterns: multiple containers in one pod referencing a single claim, multiple pods referencing one shared claim, and driver-level partitioning or time slicing, in which the vendor driver presents one physical accelerator as several schedulable devices. Each pattern changes who can reach the device, how the scheduler places workloads and what happens when one workload misbehaves.

Containers in one pod that reference the same claim share the device and the pod's trust boundary. Kubernetes does not add memory or compute separation between those containers, so this pattern fits containers that already belong to one application and may share volumes or a process namespace. Pods sharing one claim are a different case. The claim is allocated once, so the pods share the device for as long as the claim exists, and they may need to land on the same node. Isolation then depends on the driver and on what each pod can do to the device, so keep pods from different tenants or data classifications off a shared claim.

Driver-level partitioning and time slicing are where isolation claims vary most. Some hardware can split memory and compute into partitions with fixed limits, while time slicing typically switches work between contexts and may not give each workload a fixed memory ceiling or guaranteed share. Whether your accelerator supports either mode, how memory is fenced between partitions, whether a fault in one context can affect another, and how the device plugin or DRA driver exposes partitions are questions for the GPU vendor. Confirm each point for your exact hardware model and driver version, and ask for written statements on memory isolation, compute isolation and error containment before you rely on sharing for regulated data.

Some workloads need several GPUs in one placement decision, and sharing does not solve that. As an illustrative case, a vLLM deployment using tensor parallelism splits one model across several GPUs within a node, so every shard must be allocated together on the same node. Request the full device set in a single claim so the scheduler finds a node with all of them free, and avoid spreading the shards across separate claims, which can leave some allocated and others waiting. A practical design separates a shared pool for small inference jobs from dedicated whole-device claims for large models. Record the sharing mode, the vendor confirmation and the tenant boundary in your design document before the first production claim.

The key point for planning is that a higher-priority workload cannot be assumed to evict a device that is already allocated to a lower-priority workload. Ordinary pod preemption frees CPU and memory by removing lower-priority pods so a pending pod can be scheduled. With Dynamic Resource Allocation, a device held through a ResourceClaim should be treated as occupied until its holder releases it. Before you write queue or priority policy, confirm the exact preemption behavior for the Kubernetes version you plan to run in the official documentation, because this area changes between releases and you should not design around assumptions from older material.

The practical effect appears first in queues. A pending high-priority pod that needs a GPU may wait behind lower-priority jobs that hold every device, even though its pod priority is higher. Batch jobs that run for hours can hold accelerators for their full duration, and inference services that keep a model loaded hold their devices continuously. Illustrative example: a cluster with one free GPU, a training job holding the rest, and an interactive inference endpoint that must start within minutes will leave the endpoint waiting until the training job finishes or fails. Map each workload class to the wait it can tolerate before you size the cluster.

There are three main ways to plan around the limit. The first is reserving capacity: keep devices unallocated for higher-priority tiers, which protects latency but leaves hardware idle and adds recurring cost. The second is separating device pools by priority, using distinct DeviceClass definitions and node groups per tier so each tier draws from its own devices. This gives predictable access but can fragment capacity and lower overall utilization. The third is scheduling policy outside DRA, where an admission or queueing layer decides which jobs start, and stops or drains lower-priority work through its own mechanism. This adds a component your team must build, secure and operate.

Choose the approach per workload class, not for the whole cluster. Ask which workloads can be interrupted, whether batch jobs checkpoint their state, and which inference services must stay available during peak demand. Then test contention in a staging cluster that reproduces the mix of jobs, because scheduling behavior under load is hard to predict from documentation alone. In a quote, ask whether the scope includes a contention test, a written priority and pool design, and documented ownership for the policy. Expect the main cost drivers to be idle capacity for reserved devices, extra nodes for separate pools, and build and operation effort for any external scheduling layer.