OpenAI and Broadcom unveil 'Jalapeño' custom LLM-inference chip

OpenAI, in partnership with Broadcom, unveiled Jalapeño, its first custom silicon purpose-built for large-language-model inference. The chip is designed to improve performance-per-watt and lower the cost of serving models at OpenAI's scale, and it marks the company's most concrete step yet toward owning its own hardware stack rather than renting capacity built around third-party accelerators.

The strategic logic is straightforward: inference, not training, is where OpenAI burns the most sustained compute as ChatGPT and API usage grow. A chip tuned specifically for transformer inference — memory bandwidth, low-latency decoding, efficient serving of mixture-of-experts routing — can meaningfully cut marginal cost per token. Notably, OpenAI signaled it will keep buying Nvidia GPUs for heavier training runs, framing Jalapeño as a complement, not a wholesale replacement.

Competitively this follows a now-familiar playbook. Google has TPUs, Amazon has Trainium and Inferentia, and Microsoft has its Maia line; OpenAI joining the custom-silicon club via Broadcom (which also helps Google with TPU design) signals that every hyperscale AI operator now views Nvidia dependence as a margin and supply risk. For Nvidia, the read-through is mixed: training demand remains intact, but the most lucrative inference fleet at the largest AI lab may gradually migrate off its chips.

Developers on Hacker News and X largely welcomed the move as potential relief from Nvidia lock-in and high inference pricing, while cautioning that custom silicon is notoriously hard to bring to volume — software stacks, yields, and real-world utilization often lag the launch slide. The real test is whether Jalapeño ships at scale and whether OpenAI passes any savings to customers via the price cuts it has been signaling.