Papers for

image analysis teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Visual agents read ultra-high-res photos accurately by comparing image versions

ViRA: Toward Ultra-High-Resolution Visual Agents via Cross-Input Relative Policy Optimization

Abstract: Visual agents answer questions about ultra-high-resolution photos by zooming into the relevant regions. However, existing agents often answer from common sense instead of reading the evidence. Reinforcement learning (RL) on the final answer cannot correct this, because a guess earns the same reward as reading. We propose Cross-Input Relative Policy Optimization (CrossRPO), which trains agents on contrast sets. A contrast set asks one question about several versions of an image that differ only in a small region with the evidence and have different answers, so only reading the evidence solves the whole set. CrossRPO rewards solving the whole set, and we prove that it does not reinforce a set answered from common sense. We build contrast sets for training (ViRA-Train) and evaluation (ViRA-Bench) by editing one small object in images of 21 to 120 megapixels, and we photograph further sets with real changes (ViRA-Bench-Real) to exclude editing traces. We train two Visual Reasoning Agents, ViRA-9B and ViRA-27B, with CrossRPO on only about 3,500 items of ViRA-Train each. CrossRPO achieves 63.3% set accuracy on ViRA-Bench at the maximum resolution, against 59.3% for GRPO. ViRA-9B is the most accurate open model of its size on all benchmarks, and ViRA-27B outperforms Claude Haiku 5.5 and GPT-6-Luna, both at their highest thinking level, on most benchmarks.

Mon 16 NovComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Reading tiny details in huge photos is hard for AI agents because they often guess based on common sense instead of actual evidence. The authors designed a new training method called CrossRPO that makes agents compare different versions of an image with slight changes to ensure they really look at the evidence. They created special training and testing sets with ultra-high-resolution photos to teach and evaluate these agents. Their trained agents performed better than previous models at answering questions correctly by truly reading the image details.
Open → 2611.21605v1

Large language models generate medical image classifier parameters without training data

Weight-Space Programming for Medical Imaging: Do We Still Need Training Data?

Abstract: Can a large language model (LLM) write the parameters of a medical image classifier from medical knowledge alone, without any training data? Such classifiers normally require labelled images, which are costly and often unavailable. We investigate this question through weight-space programming (WSP), in which an LLM writes or adapts the weights and biases of a neural network directly, without optimisation on training data. Claude Opus 5.5 and GPT-6 Astra each generate the parameters of a small LeNet-style CNN in a single reply, without tool use and without access to any image. We compare these networks against zero-shot MedSigLIP and against the same architecture trained on labels using the Adam optimiser, across 7 classification tasks in four modalities: chest X-ray, breast ultrasound and abdominal CT (all from MedMNIST), and pelvic MRI (SynthUterus). The LLM-written networks perform above chance on all six MedMNIST tasks, with a mean test AUC of 0.719 for Opus and 0.711 for Astra. Both models outperform MedSigLIP on 5 of 6 tasks while using more than 10,000 times fewer parameters. However, the same network trained with Adam on only 256 labelled images achieves a higher AUC on every MedMNIST task (mean AUC 0.828). On SynthUterus, Opus attains an AUC of 0.764, whereas Astra falls below chance (AUC 0.136) owing to a systematic error. These results show that LLMs can write functional medical classifiers, but that they do not yet replace training data. We release the protocol, logs and all generated weights at https://github.com/MaximilianLindholz/weight-space-programming-medical.

Mon 9 NovComputer Vision and Pattern Recognition
The gist
Medical image classifiers usually need lots of labeled images to learn how to recognize diseases, but getting these images is hard. The authors tested whether large language models (LLMs) can create these classifiers just by writing down the network's settings, without seeing any images or training data. They found that the LLMs could build classifiers that perform better than a simpler method but not as well as ones trained with even a small amount of real data. This shows LLMs can help build medical image classifiers but cannot replace the need for training images yet.
Open → 2611.12066v1

Visual Jev speeds answering multiple questions about one image

Visual Jev: Accurate and Efficient Decisions from Shared Visual Context

Abstract: Many vision applications ask several independent, forced-choice questions about the same image. Visual Jev encodes the image and public context once, executes isolated question suffixes as a batch, and reads candidate probabilities from the backbone's language-model head. Across four benchmarks, answer-supervised post-training raises equal-weight macro accuracy from 70.6% to 76.1%, with the gain concentrated on the two task families represented in training. At N=32 questions per image, shared batched execution is 8.9x faster in warm amortized time than independent serial execution and remains 3.4x faster than an already-batched baseline that recomputes the prefix, at the cost of higher peak memory. A matched typed-head control offers no consistent accuracy advantage over the language-model-head readout. The supported design is therefore simple: adapt the backbone for quality, retain the existing readout, and share execution for efficiency.

Tue 22 SeptComputer Vision and Pattern RecognitionMachine Learning
The gist
Many apps need to answer several separate questions about the same picture quickly and accurately. The authors made a way to process the picture once and then answer many questions in a batch, making it much faster without losing accuracy. They also improved accuracy by refining the system using answers it already knew. Their approach uses a simple design that keeps good results while saving time, though it requires more memory.
Open → 2609.25845v1