Multilingual dataset boosts AI crisis warning messages in five languages
CrisiLangs: A Multilingual Resource for Crisis Warning Generation
Computation and Language
Summary
During emergencies, it’s important to send warning messages quickly and in multiple languages. The authors noticed that existing AI tools only handle English well for this task. They created CrisiLangs, a new set of examples for warnings in Italian, Bulgarian, German, and Greek, alongside English. They showed that training AI models on this new multilingual data helps the models write better warnings in all five languages. They also found that following official emergency guidelines in the prompts and using lower randomness in message generation improved message quality and preference.
What this means in practice
- •For emergency communication teams: Create prompt-based AI tools to generate warning messages in multiple European languages during crises.
- •For multilingual ai developers: Fine-tune language models to improve multilingual crisis message generation using the CrisiLangs dataset.
Authors
Giacomo Gonella, Stefano Menini, Marco Guerini
Abstract
Communicating with the public during an emergency entails issuing warning messages promptly. In Europe, it often requires producing them in more than one language. Recent work has shown that language models can be trained to generate warning messages as a crisis unfolds, but the available data covers only English. To fill this gap, we present CrisiLangs, a multilingual extension of CrisiText (an English dataset of crisis events paired with warning messages written according to expert communication guidelines) to Italian, Bulgarian, German, and Greek. Expert native speakers translated the warning messages of 1,000 events drawn from 411 scenarios, for a total of 5,000 event-message pairs across five languages. We test the resource to fine-tune a multilingual model on all five languages jointly, with and without the emergency guidelines in the prompt, and compare it with a zero-shot baseline. The fine-tuned models outperform the baseline on ROUGE-L, BLEU and BERTScore in every language and configuration. In a human preference study, low decoding temperatures receive the most preferences in all five languages and in both configurations, while the effect of the guidelines varies by language.