Teaching smaller AI models with 'hidden layers' does not make them smarter

Researchers tried to make smaller AI models learn more effectively by showing them the internal thought processes of larger models. It turns out this method does not consistently improve the smaller models' performance on common tasks. This means that simply exposing a small AI to the internal workings of a big AI is not enough to make it significantly better.

The idea was that if a small AI could see how a big AI 'thinks' internally, it would learn more efficiently than just seeing the final answers. This paper shows that this shortcut does not work as hoped. Companies trying to build smaller, cheaper AI models that perform like larger ones will need to find other ways to train them.