Researchers map how language models remember things: a specific circuit that retrieves information in order

Researchers found that large language models use a specialized attention mechanism called induction heads to retrieve information from context in a specific sequence — the same way humans recall lists of items. When these heads are removed, the models become significantly worse at recalling information in the correct order, suggesting this is a core mechanism for how these systems work.

This is mechanistic evidence about how language models actually process sequential information — not speculation or inference, but direct observation of which neural circuits do the work. It matters because most AI safety and interpretability work proceeds by guessing what models are doing inside. This paper shows you can identify specific attention heads and verify their function by removing them and measuring the damage. That's the kind of granular understanding that makes AI systems less like black boxes.