AI agents now tested on the one skill that matters: changing their minds when new information arrives

Researchers created a benchmark that tests AI agents on a real-world problem most existing tests ignore: keeping track of information when facts contradict each other, change over time, and come from different sources. This matters because every AI assistant deployed in the world actually encounters this problem — scattered, conflicting information that requires constant belief revision — but existing tests assume a single source of truth that never changes.

Right now, AI benchmarks measure whether agents can follow instructions in clean, static environments. They don't measure whether an agent can handle a user's workspace where emails contradict documents, older decisions get overturned by new information, and preferences emerge through corrections rather than explicit statements. This benchmark exposes a real gap: large language models show a 15% spread in performance, and the framework a company uses to build its agent matters almost as much as the model itself. An AI assistant that fails at belief revision is useless in actual deployment — it will confidently act on outdated information while missing contradictions a human would catch immediately.