AI can now grade radiology reports faster and cheaper than before — but only if you retrain it first

Researchers found that large language models can evaluate radiology reports more accurately than existing automated metrics, and that retraining a mid-sized model on just 1,300 examples makes it 25% better while running 37 times faster. This means hospitals could deploy AI systems to catch errors in radiology reports at a fraction of the current cost, without needing massive labeled datasets.

Radiology report quality matters because errors get patients sent to the wrong treatment — but manually checking every report is expensive and doesn't scale. Until now, the automated metrics hospitals used were optimized only for chest X-rays and fell apart on other imaging types. This work shows you can build a general-purpose grader that works across all imaging types and anatomies, and that you don't need massive training datasets to do it. That removes a real bottleneck in hospital workflows.