Researchers measure whether AI explanations stay consistent — or flip for no reason

A new metric now tests whether AI models give the same explanation for similar inputs, or contradict themselves on what they're looking at. This matters because an AI that claims to base decisions on one feature but actually switches between features without warning is unsafe to deploy, even if its predictions are correct.

Most AI explanations are checked one prediction at a time. Nobody was actually measuring whether an AI tells a coherent story across similar cases — or whether it just makes things up differently each time. This paper shows you can quantify that inconsistency using a simple similarity score. What this reveals: plenty of AI systems are internally contradictory. They might get the right answer but for reasons that shift moment to moment, which means you can't trust the explanation, and you can't fix the model when it fails.