Small language models can now route requests between AI agents efficiently without sending everything to the cloud

Researchers built a system that lets small AI models (3B–7B parameters) decide which specialized AI agent should handle a request, instead of treating routing as open-ended text generation. This means AI agent networks can make routing decisions locally on edge devices or private servers, cutting latency and keeping data private without sacrificing accuracy.

Most AI systems today route requests by generating text, which is slow and expensive at scale. This paper shows that treating routing as a constrained decision problem (does this need one agent, multiple agents, a direct answer, or human escalation?) lets small models do the work efficiently. The practical implication is that decentralized agent networks become feasible without bottlenecks — you don't have to send every request to a central cloud system to decide where it should go. For systems handling sensitive data or operating under latency constraints, that's a real structural change.