Researchers build a financial sentiment dataset that shows how LLMs fail — and in predictable ways

A team created a labeled dataset of 1,439 financial analyses where they recorded not just what an AI model said, but how it reasoned, where humans corrected it, and whether the prediction was right. The dataset is built for training AI systems to reason better about stocks and bonds by learning from those corrections — which means financial AI companies can now identify and fix specific failure modes instead of guessing.

Until now, AI systems trained on financial data either worked or didn't, but nobody could see exactly where the reasoning broke. This dataset makes those failures visible. The researchers found a specific failure mode they call 'Latent Reasoning Drift' — where the model confidently invents information that wasn't in the input, then bases conclusions on that invented fact. It also found systematic confidence miscalibration, meaning the model is often wrong about how sure it is. This matters because financial AI is already deployed in real trading systems and risk analysis. If you can measure a failure mode, you can target it. If you can't, you're just hoping the next training run works better.