Open Brazilian Portuguese dataset improves misogyny detection online
MISO-BR: An Open Corpus for Misogyny Detection in Brazilian Portuguese
Computation and LanguageArtificial Intelligence
Summary
Misogyny, or hate speech targeting women, is common online in Brazil, but few public resources exist to detect it in Portuguese. The authors created MISO-BR, a well-documented collection of nearly 900 social media posts labeled as misogynistic or not, using both automatic and manual methods to ensure quality. They tested traditional language models and modern transformer models, finding transformers performed much better. They also explain limitations of their dataset and provide all data and tools publicly so others can check their work.
What this means in practice
- •For social media platform teams: Improve tools to automatically identify misogynistic content in Brazilian Portuguese posts using an openly available, well-annotated dataset.
- •For content moderation service providers: Develop and test moderation systems for Portuguese-language online communities with a transparent evaluation dataset focused on misogyny.
Authors
Fábio A. S. Passos, Péricles B. C. Miranda, Ana Paula Cavalcanti
Abstract
Misogyny is among the most prevalent forms of online hate speech directed at Brazilian users, yet open and auditable resources dedicated to it in Brazilian Portuguese remain scarce. We introduce MISO-BR, an open, documented and versioned corpus of 896 balanced social media texts annotated for binary misogyny, released with its construction protocol and reproducible baselines. The corpus combines lexicon- and target-guided scraping from X with items imported from OLID-BR, a large language model triage stage whose prompt and decoding parameters are released, and manual annotation by three annotators following a written guide. Inter-annotator reliability reaches a Fleiss' kappa of 0.788 over 816 items. Under a single fixed protocol we compare four classical classifiers (Multinomial and Bernoulli Naive Bayes, a linear SVM and a random forest) with fine-tuned BERTimbau Base and Large. The classical models cluster between 0.651 and 0.676 macro-F1 and are precision-oriented but recall-poor. The transformers reach 0.833 macro-F1 and accuracy (Wilson 95% CI for accuracy [0.77, 0.88]), a gap of roughly 16 points, while Base and Large are indistinguishable on the test set despite a 2.6x difference in training cost. We argue that the value of the resource lies less in its headline scores than in the auditability of its pipeline, and we state three limits that users must reckon with: the artificial class balance, whose effect on precision under realistic prevalence we quantify; the association between provenance and label, which the released source field makes measurable; and the selection bias induced by lexicon-guided sampling and automatic triage. The corpus, annotation guide, models and experimental scripts are publicly available.