CAMO: A Class-Aware Minority-Optimized Ensemble for Robust Language Model Evaluation on Imbalanced Data
2026-04-08 • Computation and Language
Computation and LanguageMachine Learning
AI summaryⓘ
The authors present a new method called CAMO to improve how computer programs recognize categories when some categories have fewer examples than others. CAMO works by carefully combining multiple models' predictions to better support these less common categories without hurting overall accuracy. They tested CAMO on two challenging datasets with uneven category sizes and found it outperformed seven other ensemble methods across different language models. The results show that CAMO adapts well to different models and tasks, making it a useful tool for handling imbalanced datasets.
class imbalanceensemble methodsmacro F1-scoreconfidence calibrationlanguage modelszero-shot learningfine-tuningminority classesmodel uncertaintycategorization
Authors
Mohamed Ehab, Ali Hamdi, Khaled Shaban
Abstract
Real-world categorization is severely hampered by class imbalance because traditional ensembles favor majority classes, which lowers minority performance and overall F1-score. We provide a unique ensemble technique for imbalanced problems called CAMO (Class-Aware Minority-Optimized).Through a hierarchical procedure that incorporates vote distributions, confidence calibration, and inter model uncertainty, CAMO dynamically boosts underrepresented classes while preserving and amplifying minority forecasts.We verify CAMO on two highly unbalanced, domain-specific benchmarks: the DIAR-AI/Emotion dataset and the ternary BEA 2025 dataset. We benchmark against seven proven ensemble algorithms using eight different language models (three LLMs and five SLMs) under zero-shot and fine-tuned settings .With refined models, CAMO consistently earns the greatest strict macro F1-score, setting a new benchmark. Its benefit works in concert with model adaptation, showing that the best ensemble choice depends on model properties .This proves that CAMO is a reliable, domain-neutral framework for unbalanced categorization.