Punjabi song toxicity dataset created with detailed multimodal labels
The Punjabi Geet Toxicity Dataset (PGT-FG): Fine-Grained Multimodal Annotations for Toxic Content in Punjabi Songs
Computation and Language
Summary
Detecting harmful or toxic content in Punjabi songs is hard because there isn’t much data available. The authors collected over 400 Punjabi songs from YouTube and split them into short parts that they labeled for types of harmful content like violence or drug references. Four native speakers carefully annotated these parts, agreeing on whether each was toxic and the severity. This dataset can help build tools that better understand and detect toxic content in Punjabi music, considering both lyrics and other cues.
What this means in practice
- •For content moderation teams: Identify and filter toxic content specifically in Punjabi songs by using segment-level labels covering multiple toxicity types.
- •For social media platform developers: Develop systems for better detection of harmful Punjabi music content by combining language and context from audio and lyrics segments.
Authors
Lakhjeet Kaur, Jaspreet Kaur, Komal Rani, Yougal Attri, Ramakant KumarJaydeep Kishore, Pravin Kumar
Abstract
Resources for toxicity detection in Punjabi remain limited, particularly for musical content that combines linguistic and multimodal information. We present the Punjabi Geet Toxicity Dataset (PGT-FG), a segment-level resource developed to support research on toxicity detection in Punjabi musical content. The dataset was constructed from publicly available Punjabi songs collected from YouTube. An initial collection of 620 songs resulted in 465 successfully downloaded songs and 3,402 preliminary segments from 450 songs. After quality control and annotation, the final dataset contains 2,396 unique segments from 392 songs. Four native Punjabi annotators produced 2,870 annotation records, including a shared subset of 158 segments annotated by all four. Each annotation includes binary toxicity, eight fine-grained toxicity categories, severity, sentiment, mode, and an uncertainty flag. Inter-annotator agreement was measured with Fleiss' kappa on the shared subset, and majority-based resolution produced the final segment-level labels. Of the 2,396 segments, 705 (29.42%) were labelled toxic and 1,691 (70.58%) non-toxic. Violence was the most frequent category, occurring in 539 toxic segments (76.45%), followed by gang references (227, 32.20%) and drug references (133, 18.87%). The overall toxicity label achieved substantial agreement (kappa = 0.6521). PGT-FG provides a fine-grained resource for Punjabi toxicity detection and multimodal content analysis. The resolved dataset, with one final record per segment, is publicly available on Zenodo (https://doi.org/10.5281/zenodo.23040156). Original lyrics, lyric excerpts, and copyrighted audio or video content are not redistributed.