Large AI models can now adapt their internal speed-boosters as they run

Large AI models can now continuously train the part of their system that guesses the next words, doing it live as users interact with them. This means new models can be deployed faster and stay efficient even as user requests change.

Running large AI models is expensive. One trick to make them faster is to have a smaller AI guess the next words, but keeping that guesser AI smart as user patterns change is hard. This new system lets the guesser AI learn and adapt on the fly, making the whole system cheaper to operate and more responsive to real-world use.