NVIDIA CUDA 13.4 enables memory-enforced GPU sharing in Kubernetes

The weakest point of GPU sharing has always been memory. Time-slicing and MPS let workloads share compute, but one job could still grab too much memory and crash or starve its neighbors. CUDA 13.4 adds enforced per-workload memory limits, so platform teams can pack Kubernetes pods onto a GPU with guarantees about how much memory each one can use.

NVIDIA's blog frames it for platform engineers running mixed AI fleets: inference services, notebooks, embedding jobs and small fine-tunes that each need a fraction of an H-class or Blackwell device. With enforcement in the CUDA layer, schedulers can make bin-packing decisions with confidence instead of over-provisioning to avoid out-of-memory cascades.

This complements Multi-Instance GPU (MIG), which offers hard hardware partitions but in fixed sizes. Memory-enforced sharing is more flexible, though isolation is software-level. It arrives the same day AWS added MPS-based GPU sharing to ECS, and together the two suggest fractional GPUs are becoming the default for inference rather than an optimization trick.

What to watch: integration into the NVIDIA GPU Operator and device plugin, how cloud-managed Kubernetes services (EKS, GKE, AKS) expose the feature, and real-world benchmarks on latency jitter when multiple tenants share a device.