Common cues help humans guess event typicality but AI struggles
Hey, guess what: CUES lead to inferences about typicality
Computation and Language
Summary
People learn over time what events are common or unusual and use clues in conversations to figure out what others really mean. The authors studied which clues people use to guess if an event is typical or not, including everyday knowledge, how something is emphasized, and sarcasm. They tested humans and large language models (LLMs) and found that while people use these clues well, current AI systems often lack the background knowledge needed to understand if events are typical, making it harder for them to interpret these clues correctly.
What this means in practice
- •For conversational ai developers: Improve dialogue systems by integrating cues like commonsense knowledge and utterance emphasis to better infer typical or atypical events in conversation.
- •For social robotics teams: Use insights about how humans detect sarcasm and typicality cues to enhance robots’ understanding of social interactions.
Authors
Alison Chi, Sanghyeok Choi, Hannah Rohde, Emily Allaway
Abstract
Throughout life, we accumulate knowledge of how typical certain events are. We use this knowledge in daily conversation, often interpreting meaning beyond what has been said explicitly. In this work, we take steps to better understand this meaning recovery, investigating both atypicality and typicality inferences. First, we construct two corpora, EventTyp and the human-validated EventTypGold. We conduct large-scale human and LLM experiments and find that for humans, three CUES predict inferences about typicality: Commonsense Knowledge, Utterance Emphasis, and Snark. However, in most experimental settings, we find that LLMs lack the prerequisite Commonsense Knowledge necessary to accurately assess an event's general typicality, and therefore struggle to follow these CUES.