The world is being quietly rearranged by people who write very long documents.


The title they went with Freeze Deep, Train Shallow: Interpretable Layer Allocation for Continued Pre-Training Noisy translates that to

Small teams can now cheaply improve large AI models by freezing the core


Researchers found a way to update large AI models more cheaply and effectively. This means smaller teams can improve existing models without spending a lot of money or computing power.
Updating large AI models usually means retraining the whole thing, which costs a lot of money and computing power. This paper shows how to do it by only updating specific parts. This makes it much easier for smaller companies or research groups to customize powerful AI models for their specific needs.
Watch for more specialized AI models appearing from smaller companies or academic labs, built on top of existing large models.

If you insist
Read the original →