DeepSeek open-sources DeepInfer, the inference framework behind its production services

DeepSeek has published DeepInfer, the internal inference stack that powers its own API and apps. It shipped alongside deepseek-recipe, the protocol layer that turns API requests into model inputs and parses model outputs back out. Together they expose both the model-execution layer and the protocol-handling layer that self-hosted DeepSeek deployments have so far had to reverse-engineer.

The release is not a full serving platform. As the dev.to analysis notes, teams still need their own routing, autoscaling and observability. What DeepInfer offers is the engine itself, tuned for DeepSeek's mixture-of-experts architectures. Analysts quoted by Interconnects suggest this stack is what lets DeepSeek reach up to 90% gross margins on token sales, which explains why its API pricing keeps undercutting Western rivals.

The obvious comparison is with vLLM and SGLang, the open engines most teams use to serve DeepSeek weights today. Community discussion is focused on whether DeepInfer's kernels outperform those engines on V4.1-class models, and whether it will become the default way to run DeepSeek in-house. The release came days after V4.1-Flash-1017 shipped under an MIT license.

The timing is awkward. The same week, DHS ordered US federal civilian agencies to rip out DeepSeek technology, including tools built on it. Open-sourcing the serving layer makes DeepSeek easier to adopt in the private sector even as US government use is cut off. Watch for independent throughput benchmarks against vLLM and SGLang, and for how much of the stack is actually portable to non-DeepSeek models.