Visual Jev speeds answering multiple questions about one image

Visual Jev: Accurate and Efficient Decisions from Shared Visual Context

Computer Vision and Pattern RecognitionMachine Learning

Summary

Many apps need to answer several separate questions about the same picture quickly and accurately. The authors made a way to process the picture once and then answer many questions in a batch, making it much faster without losing accuracy. They also improved accuracy by refining the system using answers it already knew. Their approach uses a simple design that keeps good results while saving time, though it requires more memory.

What this means in practice

  • For image analysis teams: Answer multiple independent questions about the same image faster by sharing visual processing steps in batch.
  • For mobile app developers: Speed up image-based question answering apps by processing images once and running multiple queries together to reduce response time.

Authors

Guanxu Yu, Yuhang Yao

Abstract

Many vision applications ask several independent, forced-choice questions about the same image. Visual Jev encodes the image and public context once, executes isolated question suffixes as a batch, and reads candidate probabilities from the backbone's language-model head. Across four benchmarks, answer-supervised post-training raises equal-weight macro accuracy from 70.6% to 76.1%, with the gain concentrated on the two task families represented in training. At N=32 questions per image, shared batched execution is 8.9x faster in warm amortized time than independent serial execution and remains 3.4x faster than an already-batched baseline that recomputes the prefix, at the cost of higher peak memory. A matched typed-head control offers no consistent accuracy advantage over the language-model-head readout. The supported design is therefore simple: adapt the backbone for quality, retain the existing readout, and share execution for efficiency.