The Image Reconstruction Game: Drawing Common Ground Through Iterative Multimodal Dialogue
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceComputation and Language
AI summaryⓘ
The authors created a new test called the Image Reconstruction Game where a vision-language model guides an image generator step-by-step to improve a picture. They tested different combinations of description and generation models and found that the quality mostly depends on how well the describer gives instructions, while the generator affects if improvements help or not. Complex images like math diagrams are hardest to recreate. They also discovered that having a limited number of instructions changes how the image improves over time. Finally, the authors showed that automated judging doesn’t always match what humans prefer, so human feedback is still important.
vision-language modelimage generatoriterative refinementimage reconstructionbenchmarktoken budgetcorrection vocabularyautomated judgehuman validationspatial instructions
Authors
Sherzod Hakimov, Mattia D'Agostini, Ivan Samodelkin, David Schlangen
Abstract
We introduce the Image Reconstruction Game, a fully automated benchmark in which a vision-language model issues corrective instructions to an image generator across multiple turns, making accumulated common ground directly observable as a rendered image. Benchmarking two Describer models crossed with two Generator models across seven image categories, we find that the describer is the dominant factor in reconstruction quality, while the generator determines whether iterative refinement helps or hurts. Mathematical and geometric images pose the greatest challenge. The describer's token budget strongly affects convergence: shorter budgets yield sparser first renderings with more room for visible improvement, while longer budgets raise absolute quality but leave less to fix. Stronger describers use a richer correction vocabulary spanning spatial, numeric, and structural categories, while weaker describers concentrate on surface properties and tend to stop after a few turns. Human validation shows that the best automated judge reaches only slight-to-fair agreement with human preferences, and automated scores require human recalibration to be used reliably.