ZML launches free cross-chip inference server backed by Hugging Face founders

ZML's free inference server, ZML/LLMD, targets one of AI infrastructure's stickiest problems: hardware lock-in. It runs open-source LLMs efficiently across NVIDIA, AMD, Google TPU, Apple Metal and Intel Arc, letting teams pick hardware on price and availability rather than being tied to CUDA. Hugging Face acts as the storage layer for models, and the backer list is notable—HF founders Clément Delangue and Julien Chaumond, Docker's Solomon Hykes, and Meta's Yann LeCun.

Mechanically, cross-chip inference requires abstracting away vendor-specific kernels and memory models—hard engineering that CUDA's dominance has historically discouraged. If ZML delivers competitive throughput on non-NVIDIA silicon, it chips at the moat that keeps NVIDIA pricing high. Delangue framed it as proof 'we're still just scratching the surface of how much better, faster, cheaper we can make inference for open-source models.'

Competitively, this fits the week's cost-and-abstraction theme—Vercel's Rauch pushing model routing, Microsoft's MAI cost-cutting, NVIDIA's own Nemotron cost claims—and validates a portability layer that reduces dependence on any single chip vendor (ironic given NVIDIA's simultaneous LeRobot partnership with Hugging Face). Skeptics will want independent throughput numbers on AMD and TPU versus NVIDIA before believing the lock-in is broken. Watch adoption and whether cloud providers integrate it.