Amazon ECS adds GPU sharing with NVIDIA Multi-Process Service

Fractional GPUs are now a first-class option in Amazon ECS. With NVIDIA's Multi-Process Service, several ECS tasks can share one GPU concurrently, instead of each task reserving a whole device and leaving most of it idle.

MPS works by funneling CUDA work from multiple processes through a shared server, so kernels from different tasks can run on the GPU at the same time rather than time-slicing. That suits workloads that need only a fraction of a GPU: small-model inference, embedding generation, and autonomous-vehicle and ADAS simulation, the three use cases AWS calls out. The economic argument is utilization: a GPU running at 20% for one task becomes much better value when four tasks share it.

The move tracks a broader industry shift this week. NVIDIA detailed memory-enforced GPU sharing in Kubernetes with CUDA 13.4, and both point to inference fleets dominated by smaller models where whole-GPU allocation wastes money. Kubernetes users have had MIG and time-slicing options for some time; ECS customers have lagged until now.

Caveats: MPS historically offers weaker isolation than MIG, so a misbehaving task can affect neighbors, and memory contention is a real risk. That is exactly what CUDA 13.4's enforcement addresses on the Kubernetes side. Watch whether ECS adds per-task memory limits and how pricing guidance changes for shared instances.