The world is being quietly rearranged by people who write very long documents.


The title they went with Reproducibility and Artifact Consistency of the SIGIR 2022 Recommender Systems Papers Based on Message Passing Noisy translates that to

Research papers on AI recommenders are riddled with errors, making progress claims suspect.


Researchers checked AI systems that suggest products or content. They found many papers had bad data splits and code that didn't match the descriptions. This means the reported improvements in AI recommendations might not be real.
This paper reveals a widespread problem in AI research: bad data practices and inconsistent reporting. It suggests that many published results in recommender systems, a key area of AI, may be unreliable. This makes it harder to build on previous work and could slow down actual progress in making AI recommendations useful and trustworthy.
Watch whether future SIGIR or similar AI conference papers start including verifiable code and data that passes independent checks.

If you insist
Read the original →