OpenAI details how its Habitat storage layer serves 1B+ ChatGPT users with 12x faster reads

OpenAI's engineering blog published part two of its series on scaling Habitat, the online storage system behind ChatGPT, now serving over 1 billion users. The headline result is 12x faster reads after reworking multi-tenancy, adding global caching and building on Azure Cosmos DB.

The post covers three layers. First, reliable multi-tenancy, so one workload's spikes do not starve others — critical when ChatGPT features, memory, conversation history and agent state share infrastructure. Second, global caching to serve reads close to users and cut cross-region latency. Third, using Cosmos DB as the durable backbone, which gives OpenAI managed global replication instead of running its own distributed database.

Context: much of the AI-scaling conversation focuses on GPUs, but consumer AI products at this size are also large-scale stateful web services. Features such as persistent memory and long-running agents multiply read and write volume per user. The post is also a quiet endorsement of Microsoft's stack at a time when OpenAI has diversified compute to other clouds, including its models on AWS Bedrock.

Caveats: engineering blogs show the success path and omit cost figures. The 12x number is relative to OpenAI's own prior baseline, not a general benchmark. Still, it is a useful reference architecture for teams building agent-memory backends at scale.