Models identify American satire more accurately than Australian satire
Unreliable Narrator Framing as a Cost-effective Mechanism for Satire Explanation of Australian and American Satirical News
Computation and Language
Summary
Satirical news is meant to be funny but can sometimes confuse readers who don't get the joke. The authors explored a way to explain why satire works using a theory about unreliable narrators, which are storytellers who can't be fully trusted. They tested this idea on satirical news from Australia and the USA using affordable language models. Their study found that the models were better at recognizing satire in American articles than Australian ones. They also created a new collection of data and a way to measure how well satire explanations work.
What this means in practice
- •For fact-checking teams: Use cost-effective language models framed on unreliable narrator theory to explain and flag satirical news more accurately in American sources.
- •For social media platforms: Integrate satire explanation mechanisms that can help users understand humorous misinformation, especially from American satirical news.
Authors
Kanay Gupta, David Y. Liu, Aditya Joshi
Abstract
Satirical news is a unique form of misinformation: it is intended to be humorous by the original publication source but can be misinterpreted. Explanation of satirical news articles is challenging because recent and local context needs to be incorporated. This is a pilot study that explores explanation generation for satirical news articles from Australia and compares them alongside those from USA. The paper demonstrates that framing explanations on unreliable narrator theory on open-weights models can be a feasible, cost-effective alternative to frontier models. We find that models identify the satirical tropes more accurately on the American source than the Australian one. This work results in a novel dataset and an evaluation benchmark for satirical explanation.