AI summaryⓘ
The authors explain that large language models (LLMs) have made misinformation more complex by affecting not just the false content but the whole system that checks and fights misinformation. They propose a new framework that looks at LLMs as attackers, defenders, or weak spots within verification systems and examines different levels like the content itself, social settings, sources of evidence, and the checking processes. Using this framework, they analyze how LLMs can be used to spread or detect falsehoods and discuss current defense methods. They also point out three major challenges: improving how we measure risks across the whole system, protecting verification methods from being tricked, and creating transparent systems where humans can help verify information safely.
Large Language ModelsMisinformationVerification SystemsAdversarial AttacksDetection MethodsSocial ContextsEvidence SourcesHuman-in-the-LoopSecurity FrameworkRisk Evaluation
Authors
Lingwei Wei, Dou Hu, Wei Zhou, Songlin Hu, Philip S. Yu
Abstract
Large language models (LLMs) have transformed misinformation from a primarily content-centric problem into a broader ecosystem-level security challenge. When misused, LLMs create risks beyond false content generation, enabling attacks on the social contexts, evidence sources, retrieval corpora, and verification workflows that misinformation defense depends on. In this paper, we introduce a role-layer framework to unify these risks and defenses. The role dimension characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification workflows. Guided by this framework, we organize LLM-enabled attacks, investigate LLM-based detection and verification methods, analyze vulnerabilities in LLM-centric detection paradigms, and discuss existing countermeasures against LLM-enabled attacks. Building on this synthesis, we identify three key open challenges: moving from static detection accuracy to budgeted ecosystem-level risk evaluation, hardening LLM-centered verification pipelines against adversarial manipulation, and deploying auditable human-in-the-loop verification systems for trustworthy real-world misinformation defense.