Lightweight machine learning classifies disaster social media posts accurately
Lightweight Machine Learning for Classifying Disaster-Related Social Media Posts
Networking and Internet ArchitectureMachine Learning
Summary
It is hard to sort short social media messages during disasters into helpful categories because messages are brief and categories overlap. The authors studied simple, fast computer methods for sorting these messages and found a particular approach that balances accuracy and speed well. They also checked which categories were harder to predict and how errors relate to the model's confidence. Additionally, they explored specialized training for a tropical cyclone context, which improved results slightly and required less processing power. This work helps understand where lightweight models work well and where they struggle in disaster message classification.
What this means in practice
- •For emergency response teams: Classify social media posts during disasters to quickly identify useful humanitarian information with efficient, low-resource models.
- •For social media monitoring services: Improve real-time filtering of disaster-related content by integrating lightweight classifiers specialized for events like tropical cyclones.$Commercial implications: Enables faster and more accurate disaster content monitoring products tailored to specific event types for commercial clients.
Authors
Chengjun Jiang, Milena Radenkovic
Abstract
Classifying disaster-related social-media posts into humanitarian information labels is challenging because individual messages are typically short and context-limited, humanitarian categories can overlap semantically, and the CrisisBench label distribution is strongly imbalanced. This study investigates whether lightweight lexical machine-learning methods can provide effective and computationally efficient classification under these conditions. Using the CrisisBench v1.0 English humanitarian task, comprising 87,455 records across predefined Train, Development and Test splits, we conduct a controlled comparison of TF-IDF representations and lightweight classifiers, with Macro-F1 as the primary model-selection metric. Beyond aggregate predictive performance, the study jointly examines per-label behaviour, directional error patterns, relative decision margins, computational requirements, and the effect of domain-specific training through a tropical-cyclone specialist model. The selected balanced LinearSVC with unigram-bigram TF-IDF and C=0.5 achieved an Accuracy of 0.8282, a Macro-F1 of 0.6578 and a Weighted-F1 of 0.8315 on the held-out Test split. Lower decision margins were substantially associated with prediction errors, while the tropical-cyclone specialist achieved modestly higher in-domain aggregate performance with substantially lower feature and fitting requirements. The study contributes a controlled, label-sensitive and resource-aware evaluation of lightweight humanitarian text classification that characterises not only aggregate performance, but also where these models fail and how their behaviour changes under domain-specific training.