Apple research finds LLMs are not consistently Bayesian in their probabilistic beliefs

Apple published research on August 28 introducing a technique to study large language models as information-processing rules, quantifying how consistently they update and represent probabilistic beliefs. The headline finding: LLMs are not consistently Bayesian — they fail to update uncertain beliefs in a coherent, probability-theoretic way, exhibiting measurable internal inconsistencies.

Bayesian consistency means that when given new evidence, a rational reasoner updates its beliefs in a specific, mathematically coherent manner. Apple's method treats the model as a belief-updating function and probes whether its outputs obey the axioms of probability under new information. The discovery that they don't — systematically — has practical weight: it means an LLM's expressed confidence or probability estimates can be internally contradictory, not just noisy.

The stakes are highest in domains Apple explicitly names — medicine, science and law — where systems must represent and update uncertainty rationally to be trustworthy. A model that assigns incoherent probabilities to diagnoses or legal outcomes could mislead in ways that are hard to detect, because the outputs look confident and fluent. This connects to the week's broader caution against over-trusting agentic AI, and to Ethan Mollick's warning against ascribing coherent human-like reasoning to models based on their chain-of-thought.

Methodologically, the work adds rigor to a fuzzy but important question — can we trust LLM uncertainty estimates? — that matters for calibration, tool use and high-stakes decision support. It also fits Apple's pattern of publishing sober, limitation-focused research (echoing prior work questioning LLM reasoning) rather than capability hype. Watch whether the technique becomes a standard evaluation for calibration, whether frontier labs respond with Bayesian-consistency improvements, and how it informs deployment guidelines for AI in regulated, high-stakes fields.