The world is being quietly rearranged by people who write very long documents.


The title they went with Incompleteness of AI Safety Verification via Kolmogorov Complexity Noisy translates that to

No formal verification system can prove every safe AI system is safe


A mathematical proof shows that any verification system has a complexity limit beyond which it cannot certify safe AI behavior, even if that behavior is actually safe. This means the harder the task an AI must perform safely, the more you have to trust it without formal proof.
For years, AI safety research has chased the dream of formal verification — mathematical proof that an AI system will obey safety rules, the way engineers verify circuit designs. This paper shows that dream has a hard ceiling. A verification system works fine for simple cases, but once you ask it to verify complex behaviors, it will necessarily fail to recognize some behaviors that are actually safe. The implication is blunt: formal verification cannot be the sole safety method. It has to be paired with other approaches — testing, monitoring, restrictions on what the AI is allowed to attempt in the first place.
Watch whether this shifts AI safety research away from 'find the perfect verifier' and toward 'design systems that stay below the complexity threshold where verification breaks.'

If you insist
Read the original →