Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection
2026-06-02 • Computation and Language
Computation and LanguageMachine Learning
AI summaryⓘ
The authors worked on a competition task to detect polarization in text across many languages. They tested twelve different ways to ask the computer questions (prompts) to see which worked best. Their system was good at simply telling if polarization was present but was less accurate when identifying the specific type or how it showed up. They found prompt-based methods work well for broad detection but struggle with more detailed classification involving multiple labels or social language nuances.
Polarization DetectionMultilingual Text ClassificationPrompt EngineeringMacro F1-ScoreBinary ClassificationMulti-label ClassificationIn-Context LearningCross-Lingual AnalysisSociolinguisticsSemEval
Authors
Pritam Kadasi, Anuj Tiwari, Mayank Singh
Abstract
Our submission presented in this paper is for SemEval-2026 Task 9: Multilingual Text Classification Challenge - Polarization Detection and it covers all three subtasks: (1) binary polarization detection, (2) polarization type classification and (3) polarization manifestation identification. We adopt a systematic approach of research on short designed prompts by considering twelve designed prompts that are different in terminology clarity, detail of the definition, guidance of reasoning and in-context examples use. The experiments are conducted using aya-101 and Gemma3-27B, with the latter chosen for the submission at the end of the development through performance considerations. Our system has an average macro level F1-score of 0.762 on Subtask 1, 0.587 on Subtask 2 and 0.444 on Subtask 3 with the average accuracy of 0.819, 0.678 and 0.498, respectively, on the official test set averaged among 22 languages, respectively. With cross-task and cross-lingual analysis, we demonstrate that prompt-based approaches can be used effectively to detect coarse grained polarization but encounter more and more difficulties as far as fine-grained and multi-label sociolinguistic classification is concerned.