Control techniques improve content moderation across platforms
Control-Theoretic Content Moderation
Computers and Society
Summary
Online platforms try to keep harmful content under control, but deciding how to make moderation choices that work well overall is difficult. The authors see moderation as a system that changes over time where different actions must be balanced to meet multiple goals. They use ideas from control theory—a way to automatically adjust actions based on feedback—to create strategies that adapt moderation decisions as conditions evolve. Testing in realistic computer simulations shows these strategies perform better than simpler methods by using strict moderation only when needed and recovering quickly from sudden increases in harmful content.
What this means in practice
- •For social media platform teams: Design adaptive moderation tools that balance competing goals and respond effectively to varying harmful content levels on the platform.
- •For online community managers: Use control-theoretic moderation strategies to maintain healthier user environments by selectively applying stricter interventions when needed.
Tested on simulated data.
Authors
Benedetta Tessa, Serena Tardelli, Marco Avvenuti, Anna Monreale, Stefano Cresci
Abstract
A sizable literature studies content moderation locally, at the level of individual moderation decisions, for example by measuring or predicting the effects of specific interventions. However, the problem of how such decisions should be combined into effective platform-level moderation strategies is comparatively unexplored. We address this latter problem by formulating content moderation as a global, adaptive, and sequential decision process in which heterogeneous interventions must jointly balance multiple competing objectives. Drawing on feedback control, we introduce a general control-theoretic framework for composing moderation actions according to their expected effects on an evolving platform. We instantiate the framework in large-scale, empirically grounded simulations and compare two control-theoretic moderators against several baselines and local strategies. When moderation aims to maintain competing platform-level properties around desired conditions, the control-theoretic approaches achieve the best overall performance. They also use severe interventions more selectively and recover more effectively after external surges of harmfulness. These results demonstrate the advantages of studying content moderation as a global, adaptive, and sequential decision problem.