Agents use base models to bypass AI text detection tools
Agents Can Use Base Models to Evade AI Detection
Computation and Language
Summary
Detecting AI-generated text is important to know if a computer or a human wrote something. The authors found that AI agents can cleverly combine pieces generated by simpler base models to make text that is harder for detectors to identify as machine-made. Unlike previous methods, this stitching method keeps the meaning clear and relevant. This shows that some current AI detection tools might miss these cleverly created texts, urging improvements in detection.
What this means in practice
- •For ai safety teams: Assess the vulnerability of existing AI text detectors against responses built by agents using base models to improve detection robustness.
- •For content moderation platforms: Enhance detection algorithms to recognize AI-generated text that uses base model sampling and stitching methods to avoid detection.
Authors
Bhuwan Dhingra, Danish Pruthi
Abstract
We show that coding agents equipped with a base language model can successfully assemble responses from its samples to evade detection. Base models have been shown to evade commercial detectors, however, prior "humanization" techniques rely on using these models to paraphrase AI outputs over several iterations, which invariably results in semantic drift. In contrast, equipping coding agents to directly orchestrate the writing process by stitching text samples from a base model allows it to produce outputs that are coherent, task-specific and generally high quality. We find that Claude Opus 5 operating in a Claude Code harness effectively orchestrates a local 32B parameter OLMo-2 base LM and sacrifices little task accuracy across benchmarks spanning creative writing, factual grounding, health QA and instruction following, while using up to 90% base LM tokens. Responses constructed in this manner reduce the effectiveness of both post-hoc detectors (Pangram v4 detection rate drops from 77% to 24%) and soft watermarking applied a priori to the agent's generations (down to a simulated 10% detection at low FPR). While effective, this evasion requires a significantly larger number of input and output tokens from the agent, increasing the dollar cost per query up to 30x at API-pricing. Overall, this work demonstrates the effectiveness of a new class of adversarial attacks against AI text detection, and urges post-hoc detection providers to include outputs of base models in their training.