Content moderators can now train AI to match their judgment instead of guessing upfront what they want

A research team studied how content moderators actually define rules for AI image classifiers and built a system that lets them refine those rules as they go, instead of assuming they know what they want before training begins. The system shows borderline cases to moderators, lets them think through the gray areas, and updates the classifier based on their feedback — achieving higher accuracy than systems that ask moderators to define rules once at the start.

Content moderation has always assumed moderators arrive with a clear, stable definition of what counts as a violation. They don't. They start fuzzy and sharpen their thinking through practice — looking at edge cases, debating with colleagues, revising their standards. This paper operationalizes that real-world process into software. What changes: instead of spending weeks writing classification rules upfront, moderators can now train a classifier and refine it in real time by looking at the cases the system thinks are on the boundary. This matters because it's cheaper, faster, and produces classifiers that actually match what moderators intend instead of what they said they intended at the beginning.