AI systems explain face recognition decisions — but often make them up

Researchers tested whether large multimodal AI models can reliably explain why they match or reject faces in photos, and found they frequently generate plausible-sounding but false explanations even when their final decision is correct. This matters because biometric systems are used by law enforcement and border agencies to identify people — if the explanations are hallucinated, there's no way to audit whether the system is actually looking at the right evidence or just getting lucky.

Face recognition systems increasingly need to explain their decisions to meet legal and ethical standards, but this research shows the explanations are often fabricated — meaning the system could be matching faces for entirely wrong reasons while sounding confident. This reveals a gap between what makes AI look trustworthy (a plausible explanation) and what actually is trustworthy (explanations grounded in real visual evidence).