A mathematician proposes that AI safety might just mean making AI interpretable to humans — not actually safe

A researcher has published a mathematical theory claiming that intelligence is fundamentally about preserving similarity relations between concepts, and that AI systems may be optimized for human interpretability rather than actual safety. This means AI alignment efforts may be solving the wrong problem: making systems whose decisions look reasonable to humans, rather than making them actually behave safely in practice.

The paper argues for a reframing that cuts against the dominant assumption in AI safety: that we can build systems that are both safe and aligned with human values. Instead, it suggests AI systems may be optimized to appear interpretable and safe to human observers, which is a weaker and potentially misleading target. If this distinction is real, safety researchers have been chasing the wrong metric — designing systems that pass human inspection rather than systems that actually do what we want them to do.