The world is being quietly rearranged by people who write very long documents.


The title they went with A Formal Framework for Uncertainty Analysis of Text Generation with Large Language Models Noisy translates that to

New framework maps all the ways language models can be wrong


Researchers have created a formal system for measuring uncertainty in AI text generation — it treats the prompt you write, the text the AI produces, and how you interpret the output as interconnected sources of error. This matters because right now there's no unified way to know which part of an AI answer is uncertain or unreliable, so you can't tell if a mistake came from bad instructions, bad generation, or bad interpretation.
Until now, researchers studying AI uncertainty have worked in silos — some measuring prompt sensitivity, others measuring generation randomness, others measuring interpretation gaps — with no way to see how these sources of error interact. A unified framework that treats them as one system makes it possible to identify blind spots in uncertainty measurement and design better methods to catch AI failures before they matter.

If you insist
Read the original →