Researchers find where AI chatbots start making stuff up — and why they keep doing it

A new paper explains how the large language models that power AI chatbots produce fluent-sounding but completely false conclusions. The researchers modeled how these systems predict text as a graph search problem and found that hallucinations happen because the model either retrieves memorized patterns that ignore what you just asked it, or it creates shortcut paths through its learned knowledge that skip steps and produce plausible-sounding nonsense.

For years, it looked like hallucinations were random bugs. This paper suggests they're not random at all — they're built into how these systems learn. Early in training, the model learns to memorize facts. Later, it learns to compress multi-step reasoning into shortcuts. Both of these optimization processes directly conflict with following what you actually asked it to do. If this holds up, it means hallucinations aren't a bug you can patch out with better prompts or training data — they're a structural feature of how decoder-only Transformers work, which changes what these systems are actually good for.