NVIDIA Nemotron 3.5 Lightning lands in Amazon SageMaker JumpStart for agentic workloads

Nemotron 3.5 Lightning is NVIDIA's newest open-weights model, a 30-billion-parameter Mixture-of-Experts design that activates only 3 billion parameters per token and supports a 1M-token context window. It is purpose-built for high-volume, always-on agentic workloads — code review, security monitoring and other tasks where sustained throughput matters more than raw peak intelligence. NVIDIA claims up to 4x higher throughput and up to 30% faster task completion versus comparable options.

The AWS distribution is the news hook: the model is now available directly in Amazon SageMaker JumpStart, letting AWS customers deploy it without standing up their own inference stack. NVIDIA also published a developer deep-dive on building Nemotron 3.5 Lightning with NVFP4 quantization and quantization-aware distillation via its Model Optimizer, aimed at teams that need to hit specific latency, memory and compute targets on their own hardware.

Competitively, Nemotron 3.5 Lightning is NVIDIA's answer to the Chinese open-weight surge led by Alibaba's Qwen. Hugging Face's summer 'State of Open Models' report explicitly paired the two as evidence of a US hardware-vendor push into local inference, and noted that only about 1% of downloads now go to 100B+ parameter models — small, efficient MoE models are winning. NVIDIA is also reportedly developing a far larger Nemotron 4 family with at least one trillion parameters, potentially arriving late fall 2026.

The strategic subtext: NVIDIA selling efficient open models that run everywhere still ultimately sells more of its own silicon, whether in the cloud or at the edge. For AWS customers the practical draw is a ready-to-deploy agentic model with strong throughput economics; the caveat is that MoE models can be finicky to serve efficiently, which is exactly why the JumpStart-managed path is attractive.