Research papers on AI recommenders are riddled with errors, making progress claims suspect.
What happened
Researchers checked AI systems that suggest products or content. They found many papers had bad data splits and code that didn't match the descriptions. This means the reported improvements in AI recommendations might not be real.
Why it matters
This paper reveals a widespread problem in AI research: bad data practices and inconsistent reporting. It suggests that many published results in recommender systems, a key area of AI, may be unreliable. This makes it harder to build on previous work and could slow down actual progress in making AI recommendations useful and trustworthy.
The signal
Watch whether future SIGIR or similar AI conference papers start including verifiable code and data that passes independent checks.