The world is being quietly rearranged by people who write very long documents.


The title they went with Same Geometry, Opposite Noise: Transformer Magnitude Representations Lack Scalar Variability Noisy translates that to

AI language models count numbers backwards from how brains do


Researchers tested whether AI language models show the same noise pattern in numerical representations that biological brains do: the bigger the number, the noisier the representation, but in a constant ratio. They found the opposite. As numbers get larger, AI models actually produce more consistent, less noisy representations. This suggests AI is learning number magnitude in a fundamentally different way than animal nervous systems — using pure statistical patterns from text rather than any principle that mirrors biological information processing.
For years, researchers have noticed that some of the properties AI systems develop seem to mirror biological brains — similar geometric arrangements of concepts, similar error patterns. This paper shows at least one property doesn't carry over: how noise scales with magnitude. The biological pattern (noise scales proportionally with size, keeping a constant ratio) has been so reliable across species and contexts that it's become a benchmark for whether a system is processing magnitude in a cognitively grounded way. AI language models pass the geometry test but fail the noise test. That means either AI is solving number representation in a completely different way, or the geometry similarity is surface-level and doesn't reflect deeper alignment with how brains work. Neither is surprising, but the measurement is precise and the gap is now quantifiable.
If future research finds that this backwards noise pattern predicts real-world failures in numerical reasoning — say, AI systems making systematic errors on budgeting, forecasting, or dosage calculations where magnitude uncertainty matters — then the noise signature becomes practically important, not just theoretically interesting.

If you insist
Read the original →