Cheap models match advanced AI in content moderation accuracy and cost
Two Uncertainties Are Not One: Disagreement versus Decision-Risk in Cheap Structured Content Moderation
Computation and LanguageComputational Engineering, Finance, and ScienceLogic in Computer Science
Summary
Content moderation often involves deciding whether to keep or remove posts, which is tricky because human opinions vary and mistakes are costly. The authors studied a low-cost AI model that makes fast, simple decisions and found it performs about as well as advanced, expensive models. They also discovered that the model’s confidence mixes up two different uncertainties: when humans disagree and when wrong actions have big consequences. By splitting decisions into smaller parts, the model can better spot disagreement, but this only helps weaker models and depends on the type of content.
What this means in practice
- •For content moderation teams: Use cost-effective AI models to classify posts with accuracy comparable to cutting-edge systems, saving resources on moderation decisions.
- •For social media platform engineers: Improve moderation tools by separating confidence in decisions into human disagreement and risky mistakes to refine escalation strategies.
Authors
Karan Yadav
Abstract
Content moderation compresses a graded, contested human judgment into a binary act (leave up / take down) over billions of items, at a scale where running a frontier LLM on every post is uneconomical. A new class of frozen "System One" models returns typed, calibrated decisions instead of text at ~USD 0.000025 per call and ~100 ms. We run the first independent audit of one such model on 142,658 human-labeled comments, and benchmark it against 14 frontier reasoning models (the GPT-5.x, Claude, Gemini 3, and Grok families) on a shared probe set. Two findings stand out. First, the cheap model is accuracy-competitive: its zero-shot classification AUROC (0.83) is not statistically separable (paired bootstrap, n=56; a failure to reject parity, not proof of equivalence) from 13 of the 14 frontier systems while costing 1-3 orders of magnitude less, and it is no worse calibrated than several of them. Second, "confidence" conflates two uncertainties that come apart: human disagreement (where annotators split on satire and coded language) and decision risk (where the automatic action is costly-wrong). Under an asymmetric moderation loss these call for opposite escalation signals, so no single confidence number serves both. Because the model prices extra questions at nearly zero, decomposing the judgment into parallel sub-questions and reading their spread recovers a human-disagreement signal that the model's native confidence misses -- but this remedy is specific: it helps the weakest, disagreement-blind models and only when the decomposition aligns with the sources of disagreement; a specialist toxicity model and the strongest LLMs do not need it, and the effect is distribution-sensitive. We will release the instrument, the cached runs, and the analysis.