You can now train giant AI models on a handheld gaming device — if you compress the weights the right way

A new training method compresses the dense weight matrices inside AI models using mathematical factorization, reducing memory use by up to 199x per layer without materializing the full dense matrix at any point. This means you can train a 70-billion-parameter model on a Steam Deck with 7.2 GB of RAM instead of needing 1,245 GB — making it feasible to run full model training on consumer hardware rather than specialized clusters.

For years, training large language models required expensive GPUs or TPU clusters because the sheer memory footprint of storing and updating all the weights was the limiting factor. This method breaks that bottleneck by keeping weights in a compressed spectral form throughout training, which means a researcher or small team can now iterate on model architectures without renting cloud compute by the hour. The catch: rank-sweep experiments show that learning rate matters far more than the compression level, meaning the real bottleneck has shifted from memory to something else — and that's useful information for where to optimize next.