Conditional Reliability of Toxicity Signals for Multilingual and Code-Mixed Abuse Detection
Computation and LanguageArtificial Intelligence
Summary
The authors studied how tools that detect toxic language often fail when dealing with mixed languages, slang, or transliterations common in Indian short texts. They created ToxGate, a system that carefully weighs different toxicity signals based on the context before making a decision. Testing on multiple datasets and models, ToxGate improved toxic content detection, especially for serious abuse like slurs and threats. The authors suggest moderation systems should use external toxicity indicators as flexible clues instead of fixed truths for better accuracy.
Authors
Indraveni Chebolu, Rohan Singh, Arnab Mallick, Harmesh Rana
Abstract
Moderation systems increasingly rely on external toxicity tools, but those tools are unreliable under code-mixing, transliteration, slang, and language mismatch. We study the \emph{conditional reliability} of toxicity priors in Indian multilingual and code-mixed short text: English toxicity, Indic abuse, and rule-based severity cues can be useful evidence, but only in some linguistic and abuse-severity contexts. We propose ToxGate, a trust-fusion head that conditions each auxiliary signal on the encoder representation before adding it to the prediction state. Across three short-text abuse datasets, four transformer encoders, and five seeds per setting, ToxGate improves over matched plain encoders in 10 of 12 in-domain settings and 7 of 8 transfer settings. The largest and most interpretable gains occur in high-risk moderation slices, including explicit slurs, violent threats, and cross-dataset transfer. The broader lesson is that moderation systems should treat external toxicity tools and priors as conditional evidence rather than fixed features or ground truth, in focused ablations, source-specific gating gives the strongest results in transfer, severe-abuse slices, and high-risk triage.