AI summaryⓘ
The authors studied how well computer programs can spot false information created by AI on social media, especially when the false posts are changed to trick these programs. They created a special testing method called BiBiR that repeatedly tries to break and then improve these detectors by making small, tricky changes to posts. They found that some techniques, like rewriting posts from different voices or languages, fooled detectors most of the time without changing the original meaning. Their best detector used a method called dynamic anchor switching and did better than a usual baseline model. However, the authors note that it can still be hard to tell if changes really keep the original false meaning or not, which matters for fair testing.
machine-generated disinformationlarge language models (LLMs)adversarial attacksBuild it, Break it, Repeat (BiBiR)back-translationpersona-based rewritingtriplet contrastive learningdynamic anchor switching (DASS)label flip rate (LFR)semantic preservation
Authors
Kevin Thomas, Milosz Kasprzyk, Reuel C Igbokwe Onuigbo, Elliott Pert, Cameron Tovey, João A. Leite, Olesya Razuvayevskaya, Carolina Scarton
Abstract
Detecting machine-generated disinformation on social media is increasingly difficult as large language models (LLMs) make it easier to generate and rewrite misleading content at scale. Static benchmark evaluations, measuring detector performance on fixed held-out datasets, do not capture how detectors behave when posts are deliberately transformed to evade classification. This paper adapts the Build it, Break it, Fix it framework into Build it, Break it, Repeat (BiBiR): iterative sessions designed to stress-test detectors' robustness under iterative adversarial conditions, evaluating whether models remain reliable when disinformation posts are systematically transformed to evade classification. Across five iterations, the findings show that the best adversarial breakers' transformations came from a combination of back-translation and LLM persona-based rewriting, with the best performing technique achieving a 95% label flip rate (LFR), whilst still preserving the meaning of the original posts. The best builders' model was a triplet contrastive model with a dynamic anchor switching (DASS) architecture, which achieved an average accuracy of 72.68%, outperforming the strong baseline (a fine-tuned e5-small-LoRA) by 15 percentage points on the most robust set of breakers' adversarial attacks. The results demonstrate that an iterative framework best exposes detector weaknesses and pushes robustness improvements; however, it may still require semantic preservation analysis to distinguish valid adversarial evasion from transformations that changed the original disinformation claims' meaning.