Agentic model improves detection of intent behind online conspiracy posts
Agentic Detection of Online Conspiracies
Computation and LanguageMachine Learning
Summary
Detecting conspiracy theories on social media is hard because people might share content that looks similar but means very different things, like joking or criticizing. The authors show that understanding the social context around a message helps figure out the true intent behind it. They built a smart system that asks relevant questions about the surrounding conversations and events, making it better at spotting real conspiracy support. They tested this on a large set of Hebrew tweets and found their approach worked better than just looking at the text alone.
What this means in practice
- •For social media content moderators: Improve identification of genuine conspiracy endorsements by incorporating social context and intent inference into moderation tools.
- •For online platform trust and safety teams: Develop more accurate systems for assessing misleading or harmful posts by enabling adaptive querying of social signals around content.
Authors
Lior Biton, Oren Tsur
Abstract
Conspiratorial discourse on social media is not always expressed through explicit claims or stable lexical markers. The same surface content may express endorsement, legitimate concerns, criticism, satire, or mockery. The main challenge is therefore not only recognizing conspiracy-related claims, but inferring the speaker's intent -- the utterance's illocutionary force. We argue that this can be achieved through the use of relevant social contexts and propose an agentic framework, equipped with a set of tools supporting social queries. We demonstrate the benefits of our approach on a unique dataset of Hebrew tweets, covering 80\%--90\% of the public Hebrew tweets published over a four-year span (late 2018-- early 2023), encompassing several election cycles as well as the COVID pandemic years and related vaccination campaigns. This extensive coverage can be used in recovering different social contexts. Evaluating our framework on a manually-annotated adversarial dataset, we find that context-aware workflows consistently outperform text-only classification and that the agentic framework performs significantly better than other frameworks and settings, including a non-agentic model exposed to the same contexts available to the agent. We further provide an analysis of the results, the errors and efficiency (token economy) tradeoffs. These findings support viewing the task of conspiracy detection as a socially embedded interpretation task, in which effective classification depends not only on access to contexts, but also on adaptive reasoning in which the agent uses tools on a per-case basis, asking only for evidence relevant to its current reasoning step.