SageMaker AI real-time endpoints add NVIDIA MIG GPU partitioning

Amazon SageMaker AI real-time endpoints now support NVIDIA Multi-Instance GPU (MIG) partitioning. A single GPU can be divided into up to seven isolated partitions, each with its own memory and compute slice.

MIG is a hardware feature, not software time-slicing, so each partition gets guaranteed resources and fault isolation. That makes it practical to run several models side by side without one noisy workload starving the others. The obvious use is hosting many fine-tuned variants or small and mid-size models that would otherwise each waste most of a full GPU.

The feature fits a broader push toward cost efficiency. AWS this week also published a five-stage FinOps playbook for Bedrock costs, and the community is talking about a 17x output-price gap across models, pushing teams to right-size inference. GPU scarcity makes packing more models per accelerator directly valuable.

Caveats: MIG partitions are fixed sizes, so very uneven workloads may still waste capacity, and partitioning only works on supported GPU generations. Teams should benchmark latency per partition before migrating production endpoints.