Vision language models struggle with visual tricks in plagiarism detection
Evaluating the Semantic-to-Geometric Gap in Adversarial Defenses Against Vision-Language Model-Based Plagiarism
Computer Vision and Pattern Recognition
Summary
Students can cheat by submitting pictures of problems, fooling advanced vision-language models (VLMs) used to detect plagiarism. The authors tested simple image changes to see if they could confuse these models without making the images hard for humans to understand. They found that while VLMs are quite strong, all of them can be tricked by these visual changes. This means educators can use such tricks as a short-term way to spot cheating, but over time they will need to rethink how they design tests to keep up with improving AI.
What this means in practice
- •For educational software developers: Create anti-cheating tools that use visual modifications to reduce vision-language model effectiveness in detecting plagiarized graphical answers.
- •For security teams in online testing platforms: Incorporate visual adversarial perturbations into test materials to hinder automated plagiarism detection by vision-language models.
Authors
Christopher Burger, Christina Trotter, Joseph Carlisle, Charles Walter
Abstract
The rapidly advancing capabilities of vision-language models (VLMs) present a systemic challenge to academic integrity. VLMs now allow students to bypass meaningful engagement by capturing and submitting graphical problems as singular images, a practice we define as trivial plagiarism. To provide educators with actionable data on VLM limitations, we investigate the efficacy of heuristic adversarial image transformations designed to degrade model performance while remaining human-interpretable. Through a two-phase evaluation of introductory assessments, we manually assess baseline VLM performance on circuit diagrams, followed by an automated large-scale evaluation of topological structures (logic gates) and coordinate geometry (Karnaugh maps). We find that while highly capable VLMs can exhibit appreciable robustness, all models suffer vulnerability to adversarial perturbations. We conclude that while visual perturbations act as a viable near-term stopgap, long-term assessment security requires educators to reapproach assessment design given continually increasing VLM performance.