Neural networks learn in sudden jumps — and mathematicians finally see why

Researchers discovered that the abrupt moment when neural networks stop memorizing and start generalizing is a phase transition, like water turning to ice. This means the learning process isn't gradual — it's governed by geometric properties of how gradients flow through the network, which explains why overparameterized networks (those with far more capacity than needed) can suddenly become useful after appearing useless for a long time.

For years, the grokking phenomenon was a mystery: neural networks would memorize training data perfectly while staying useless on new data, then abruptly flip to generalizing well. Understanding it as a phase transition rooted in gradient geometry, not network size, changes how researchers think about what makes networks trainable. This could eventually speed up how we design networks or diagnose why training stalls, though right now it's still in the explanation phase.