Papers for

marketing analytics departments

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Large language models detect hidden stories in social messages without training

Zero-shot narrative detection in social messaging

Abstract: This study investigates the zero-shot ability of large language models (LLMs) to identify and classify hidden narratives in social messages. Our research hypothesis is that LLMs' extensive contextual knowledge allows them to interpret messages on a deeper, pragmatic level, going beyond basic sentiment or topic analysis. Experiments on the Dipromats and SemEval datasets show that providing models with human-written narrative descriptions significantly improves performance, without the need of training examples. In contrast, automatically generated descriptions or the use of few examples (few-shot) often degrade accuracy due to subtle shifts in framing. The study also finds that ensemble methods, particularly majority voting, enhance robustness and that larger models perform best while also being less sensitive to prompt variations. The findings validate that LLMs can effectively detect strategic narratives in a zero-shot setting, and when combined with simple ensembling and human-written descriptions, they can rival supervised systems, offering a scalable solution for narrative detection, specially when there is no training data for the vast majority of domains.

Tue 15 SeptComputation and Language
The gist
Finding hidden stories or messages in social media posts is hard, especially without prior examples to learn from. The authors show that big language AI models can spot these hidden narratives by using human written descriptions, even without training on those examples. When the descriptions are generated automatically or few examples are used, performance drops. Combining several model outputs improves results, and bigger models work best and are less sensitive to instructions. This suggests a new way to identify complex message meanings in many areas without needing lots of training data.
Open → 2609.17310v1