Model improves detection of toxic language in gamer chat messages

In-game Toxic Detection: Bi-directional Representations with Attention Residuals

Computation and LanguageArtificial IntelligenceMachine Learning

Summary

Toxic language in gamer chat—like insults or mean words—is tricky to spot because players use lots of slang and abbreviations. The authors created a new detection model called Bi-directional Representations with Attention Residuals (BRAR) that understands the whole chat context better than older methods. They tested BRAR on real in-game chat data and found it works best for identifying toxic language. This model helps make online gaming conversations safer by catching harmful messages more accurately.

What this means in practice

  • For game developers: Integrate BRAR to detect and filter toxic player messages in live game chat to improve community health.$Commercial implications: Enables creation of commercial in-game moderation tools that better handle gaming slang and short toxic messages.
  • For social media platform moderators: Adapt BRAR to analyze short, slang-heavy messages for toxicity detection on social platforms beyond gaming.

Authors

Yuanzhe Jia

Abstract

In-game toxic language has emerged as a critical concern in the gaming industry and community. While several frameworks and models for online game toxicity analysis have been proposed, detecting toxicity in player chat utterances remains a formidable challenge: stemming not only from the extremely short length of such utterances but also from the heavy reliance on game slang, abbreviations, and domain-specific jargon, which generic language models are poorly suited to recognize. This paper presents a shared task for in-game toxic language detection built upon real-world in-game chat data, and proposes the best-preforming model for the toxic language slot filling: Bi-directional Representations with Attention Residuals (BRAR). Experimental results demonstrate that BRAR effectively captures the global context and outperforms the existing baselines on slot filling.
View PDFOpen arXiv