AI summaryⓘ
The authors found that vision-language models often mess up when asked to notice changes in images, even though the image data is there. They noticed that correct answers usually include clear words about objects, colors, or positions that prove the change, while wrong answers often don’t. To fix this, they made a simple checking system called SAVER that looks for these clues in the model’s responses and asks the model to try again only if clues are missing or don’t match. This method improved accuracy a lot on several tests, mainly by helping the model better explain what it sees, rather than just retrying. Their tests also showed that the checking system itself is the key improvement, not just asking the model again.
Vision-language modelsVisual change reasoningVision encodersVerbal evidenceChange detectionSAVER methodStructured repromptingCLEVR-Change benchmarkExpression failuresError recovery
Abstract
Vision-language models (VLMs) frequently fail at visual change reasoning, even when their vision encoders contain sufficient information. We observe that correct VLM outputs tend to contain explicit verbal evidence (object names, colors, spatial locations) that supports the claimed change, while incorrect outputs often lack such evidence. We propose SAVER (Selective Auditing of Verbal Evidence for Error Recovery), a lightweight, rule-based method that parses VLM responses for this evidence and triggers structured reprompting only when evidence is missing or inconsistent. Across three change detection benchmarks and four VLMs, SAVER significantly improves accuracy on tasks where errors stem from the model failing to articulate what it saw (expression failures), with gains up to +25.8% on CLEVR-Change. The evidence patterns can also be generated by an LLM in a single call, matching the hand-tuned gate on CLEVR-Change. Ablation experiments confirm that the evidence gate, not reprompting alone, drives the improvement.