Context improves detecting lies in online hidden role game messages
Looking Into The Past: The Impact of Context on Detecting Deceptive Statements
Computation and Language
Summary
Deception detection usually looks at individual messages without considering what came before. This paper shows that when previous dialogue from a hidden role game called Avalon is included, detecting lies becomes much more accurate. The researchers found that large language models improved their accuracy by up to 31% when they had access to the earlier conversation. This means that understanding context from earlier statements helps identify deception better.
What this means in practice
- •For game developers: Enhance AI player behavior by detecting deceptive communication using prior game dialogue context.
- •For fraud detection teams: Improve automated detection of deceptive claims by incorporating prior communication context into analysis.
Authors
Alex Clay, Ernesto Jimenez-Ruiz, Pranava Madhyastha
Abstract
Deception detection typically considers messages in isolation. Previous works in claim verification and role detection have displayed that additional information such as gameplay history is crucial to successful predictions. In this work, we demonstrate the value of preceding gameplay dialogue when applied to deceptive message detection within the hidden role game of Avalon. The findings show that LLMs can see improvement across a number of metrics, including accuracy by up to 31\% with the addition of gameplay context for the target message.