AI video system learns to pick key frames by hunting for clues that answer the question

Researchers built a smarter way to select the most important frames from long videos when feeding them to AI language models. Instead of picking frames based on general importance, the system now picks frames that specifically contain clues needed to answer a particular question — cutting computational cost and letting models handle longer videos without running out of memory.

Long-form video understanding has been stuck: AI models can only hold so much information at once, so they have to choose which frames to actually look at. Existing systems pick frames either by guessing what's semantically important or by brute-force testing every combination, both of which miss the actual clues a question needs. This approach reverses the problem — it asks what information the question itself demands, then hunts for frames containing that information. That's a structural shift from guessing importance to proving relevance. What becomes possible: multimodal AI that can handle hour-long videos instead of five-minute clips, with lower computational cost. What stays stuck: the underlying bottleneck of context length in language models, which this only works around rather than solves.