A robot model learns to point at things — and then solves real manipulation tasks without retraining

Researchers built a visual reasoning model that learns to identify and point at objects in images, then uses that skill to guide a robot arm through real-world tasks. The model works across different robot hardware and improves performance on physical manipulation by 62% over previous approaches, with no task-specific retraining needed.

The core problem in embodied AI is that robots trained on one task or hardware rarely work on another — the gap between what a vision system sees and what a robot arm can actually do has been expensive and slow to bridge. This work suggests that an intermediate step — teaching a model to reliably identify and point at objects — can be that bridge, reducing the data and compute needed to deploy robots on new tasks. If the zero-shot results hold up in real deployments beyond the eight tasks tested, this cuts the cost of adding new robot capabilities from custom retraining to inference-only.