Math for training AI just got cheaper to compute — no more matrix inversions required

Researchers found a way to train statistical models on curved parameter spaces without inverting massive matrices, which normally costs a lot of compute. The method works by approximating the inverse incrementally instead of calculating it from scratch, which means training certain types of AI models becomes significantly faster.

Natural gradient methods are standard in machine learning because they converge faster than regular gradient descent, but they require computing an inverse of the Fisher information matrix, which is expensive and gets worse as models scale. This paper removes that bottleneck by maintaining a running approximation instead. The practical effect: models constrained to specific mathematical structures (like probability distributions that must stay positive, or parameters that must stay orthogonal) can now be trained without the computational penalty that made those constraints impractical before.