AI code-fixing systems can pass tests they weren't trained on—but still break the code

Researchers studied a widespread blind spot in AI systems that fix code bugs: the systems pass auto-generated tests but fail on cases the tests don't cover. This matters because code-fixing AI is already deployed in production, and it's being tested against its own generated test cases, which means nobody catches when it breaks things that matter.

Every AI system that fixes code is now being evaluated on tests it helped create or refine. This is circular—like grading your own homework. The study shows this isn't a minor problem; systems systematically overfit to narrow test cases, which means they pass validation but ship broken code into real repositories. As more code fixing moves to AI, this gap between 'passes tests' and 'actually works' becomes a production risk. Right now, nobody's measuring how often deployed AI code-fixes fail in ways the tests missed.