The world is being quietly rearranged by people who write very long documents.


The title they went with How Transformers Learn to Plan via Multi-Token Prediction Noisy translates that to

AI models can learn to plan by thinking backward from the goal


AI models can now learn to plan better. They do this by predicting multiple steps at once, which helps them work backward from a goal.
AI models have struggled with complex planning. They usually just predict the next word or action. This paper shows a different training method. The AI predicts several steps at once. This helps it learn to plan by working backward from a goal. Future AI systems could use this to solve problems that need more strategic thinking.
Watch for new AI models to adopt this multi-token prediction method, especially those built for complex reasoning.

If you insist
Read the original →