Models learn by fooling stronger models to improve performance
Uncovering the Turing Test's Latent Signal: Indistinguishability as a Reward
Artificial Intelligence
Summary
The paper shows that a smaller model can get better by trying to fool a bigger, smarter model into thinking its answers come from the bigger one. The smaller model learns through reward from how often it tricks the bigger model, without needing correct answers. This approach was tested in reasoning, math, programming, and database tasks, improving results across all of them. The researchers found that being mistaken for the teacher model often means the answer is at least partially correct.
What this means in practice
- •For ai developers: Train smaller AI models by using larger models as teachers without needing ground-truth labels for supervision.
- •For automated tutoring platforms: Improve question-answering and reasoning modules by rewarding answers that mimic expert-level responses.
Authors
Emanuele Rimoldi, Riccardo Neumarker, Tomaso Poggio, Daniel Mitropolsky
Abstract
Can a model learn by trying to fool another model? We show that it can. Inspired by the Turing Test and its recent generalization to arbitrary distinguishers, we introduce a domain-based teacher-Turing-Test, in which a stronger teacher model serves as both the target of imitation and the distinguisher, and a weaker student model is trained to make the teacher attribute responses to itself. We use this self-attribution (or fooling) probability as a reinforcement learning reward, without ever providing the student ground-truth answers, nor requiring the teacher to generate responses. We test our approach independently in four domains with verifiable answers: CommonsenseQA (reasoning), GSM8K (mathematics), KodCode (programming), and Spider (SQL), training a Qwen2.5-1.5B student/imitator model using DeepSeek V4 Flash as teacher/distinguisher. Beyond improving at passing the Turing Test, the student improves on independent, held-out evaluations in all domains and runs. To probe why being mistaken for the teacher can improve capability, our analyses reveal that self-attribution is associated with correctness -- substantially more than formatting changes -- and that it rewards partial correctness. Our results demonstrate a new, simple teacher-student training paradigm with a private and/or high-cost teacher. They also suggest that indistinguishability need not merely measure behavior -- it can also be a mechanism for learning.