NVIDIA Nemotron 3.5 Lightning MoE Lands in Amazon SageMaker JumpStart

NVIDIA's Nemotron 3.5 Lightning targets the operational reality of always-on agents: cost and throughput. The open 30B mixture-of-experts model activates just 3B parameters per token, delivering up to 4x higher throughput and 30% faster task completion for high-volume agentic workloads. Its arrival in Amazon SageMaker JumpStart gives AWS customers a ready-to-deploy option.

The MoE architecture is the key mechanism: by activating only a small fraction of parameters per inference, Nemotron 3.5 Lightning keeps serving costs and latency low while retaining a larger model's knowledge capacity—ideal for agents that run continuously and process large request volumes. JumpStart integration means one-click deployment with SageMaker's managed infrastructure.

Competitively, this is NVIDIA reinforcing that it's not just a chip vendor but a full-stack AI provider—open models optimized for its hardware, distributed through hyperscaler marketplaces. It fits the week's efficiency theme alongside LiquidAI's LFM2.5-DSpark (claiming up to 3.2x faster inference) and Alibaba's edge-focused Qwen.

NVIDIA also shipped TensorRT Model Connect in public preview, letting developers take a Hugging Face model to end-to-end TensorRT inference in two commands without ONNX export. The strategic thread: make NVIDIA's ecosystem the path of least resistance for deploying efficient agents. The caveat is benchmark verification—4x throughput claims depend heavily on workload. Watch adoption of Nemotron in production agent stacks versus open rivals.