First benchmark tests whether AI can write code for real industrial problems across multiple languages

Researchers created a test suite with 579 coding problems drawn from actual finance, aerospace, and automation work — spanning MATLAB, Python, C++, and Stata — to measure whether AI models can handle the messy variety of real industrial coding, not just synthetic textbook problems. The best AI model (Claude 4.5 Opus) solved 68% of individual problems and 42% of full industrial scenarios, revealing a gap between lab performance and what companies actually need.

Until now, AI code-generation benchmarks tested single languages and domains in isolation — they measured whether AI could solve clean, academic problems, not whether it could survive contact with the chaos of actual industrial code. This benchmark forces AI vendors to show whether their models work across the messy reality: finance code in one language, automation in another, aerospace in a third. The gap between solving individual problems (68%) and full scenarios (42%) is the real signal — it shows that industrial code isn't just harder, it's differently hard, and existing AI models degrade sharply when they have to integrate across domains and languages.