Researchers reveal AI-generated code differs in hidden ways despite similar success rates

A new benchmark shows that large language models creating their own software tools can achieve the same task completion rate while producing libraries with dramatically different quality — some with 18% more bugs, redundancy, and safety problems than others. This matters because today's evaluation methods only measure whether the code works for its immediate task, the same way a bridge inspector would only check if cars cross it without measuring whether the structure will last.

Right now, AI systems are being deployed to write and maintain their own code tools in production workflows, but nobody's actually checking the quality of what they're building — only whether it completes the assigned task. This research shows that two systems can look identical by the metrics we currently use, while one produces fragile, redundant code and the other produces stable, reusable tools. The practical risk is that AI-generated code libraries are piling up technical debt, regression bugs, and security gaps that won't show up until they're relied on at scale. What becomes visible here is a measurement gap: we need to start treating AI-generated code as a software artifact that needs maintenance, not just a temporary task-completion engine.