Summary
Because pictures contain so much detail, describing them accurately in text is hard—especially when small or complex details get missed or misstated. The authors present ZoomCap, a method that first drafts a basic caption and then zooms in only on complicated parts to gather more precise information, like reading text or spotting small objects. This approach merges new facts carefully to make captions both detailed and reliable. The authors also created FactCycle, a tool that checks each caption claim against the original image and a reconstructed image to spot mistakes. ZoomCap’s detailed and accurate captions help improve the training of AI systems that understand images and text.
What this means in practice
- •For document processing teams: Improve OCR quality monitoring with captions that focus on text details and spatial arrangement for better automated document analysis.
- •For medical ai engineers: Enhance medical visual question answering by pre-training models on images captioned with ZoomCap’s detailed evidence-based approach.
- •For computer vision developers: Generate reliable, fine-grained image captions by dynamically zooming and retrieving evidence to improve dataset quality for vision-language training.$Commercial implications: Enables products that provide precise visual descriptions from images for training or real-time applications.
Authors
Hao Shi, Jiafeng Wang, Yuzhuo Fu, Tao Zhang, Jian Bai, Juelu Zhang, Qing Cui, Chao Huang
Abstract
Detailed captions transfer visual information into vision-language model supervision, but quality is easily conflated with length: long single-pass outputs still miss small objects, text, and spatial relations or promote uncertain observations into confident claims. We formulate fine-grained captioning as recoverable visual information transfer, in which a teacher acquires missing evidence under a finite observation budget and converts it into reusable supervision. ZoomCap uses a draft caption to route conditional knowledge retrieval, OCR-sensitive reading, coverage-gap recovery, and adaptive visual zooming, allocating deeper crops only to the most complex eligible region, deriving spatial relations from grounded geometry, and fusing non-duplicate evidence. We also introduce FactCycle, which scores caption-derived atomic claims on two blind anchors--source image (truthfulness) and reconstruction (realizability)--separating fabrication from generator loss claim by claim; human and multi-judge comparisons complement this fact-level view. Against Qwen3.5-397B-A17B captions, ZoomCap achieves an 81.9% human tie-adjusted preference score with a teacher over 10x smaller, and FactCycle counts 52.4 transferred claims per image vs. 31.5 for ScaleCap. The evidence also transfers: in continual pre-training, ZoomCap-450K improves the 11-benchmark average by 1.31-2.38 points over the strongest competing corpus and Ling-Tiny-VL-v3 gains 1.4 points on a production document/OCR monitor suite (1.5 on perception) at 60B tokens; in medical pre-training, 2.64M ZoomCap-captioned images improve five VQA benchmarks by 3.04 points (vs. 1.76 for same-image Gemini-3.1-Pro captions).