Structural clues accurately spot AI-written commercial web content
SlopShape: Identifying AI-Generated Commercial Web Content
Computation and Language
Summary
Trying to spot AI-written text can be tricky because simple word-level checks fail when the text is rewritten. The authors show that looking at the way content is structured—like its order, style, and evidence—can reveal if AI wrote it, even when the text is changed. They tested this on thousands of commercial blog posts and could reliably tell AI from human writing and even guess which AI created it. Their approach is very accurate and works even when the AI text is rewritten by its own model.
What this means in practice
- •For web content moderators: Identify suspicious AI-generated commercial blog posts based on structural writing patterns to improve content authenticity.
- •For digital marketing teams: Verify if competitor or partner blog posts are AI-generated using detailed structural signals for better content strategy decisions.
Authors
Jochen Madler
Abstract
Word-level detectors identify unedited AI-generated text almost perfectly, but the literature documents their brittleness under rewording, and a word-level score neither characterizes a text nor identifies which AI model wrote it. We ask whether AI-generated text can be identified one level deeper, from structural signatures: how information is presented, in what order, with what evidence, and in what voice. We replicate StoryScope (Russell et al., 2026), which showed such patterns for AI-generated fiction, on commercial content: 2,250 pre-ChatGPT human blog posts from 268 company domains against 11,250 AI mirrors from five frontier models. A 214-feature instrument, applied by an LLM and validated in a human gold-annotation session (human-human kappa 0.928, human-model 0.946), detects AI posts from its 187 structural features alone at 98.0 macro-F1 on held-out companies, unchanged (98.1) when every AI post is reworded by its own model. The signal characterizes and attributes: AI posts share a tidy, self-announcing shape, 79.3% are attributed to the correct source against a 16.7% chance rate, and human posts occupy rare structural configurations. All effects replicate StoryScope's, consistent in direction and larger in magnitude. We release pipeline, instrument, prompts, code, and aggregate artifacts.