Papers for

automated image analysis teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Self supervised signals miss encoder decline in image training

Improving Signals, Declining Encoder: A Known-Answer Test of Label-Free Monitoring in Self-Supervised Learning

Abstract: Self-supervised training is monitored without labels, through the loss and statistics of the learned features. We report a case where we know the encoders lost part of what they had learned, so we can check whether these signals noticed. We trained LeJEPA encoders on synthetic images with known generating factors. In nine of ten seeds, the readability of those factors fell abruptly and stayed down, nearly always along the same direction in factor space. Meanwhile the loss kept falling, and most of the eleven label-free signals we tracked kept improving and rated the declining runs better than runs that did not decline. Standard change-point detection rarely flagged the decline near its onset, and selecting checkpoints by these signals mostly chose ones after it.

Mon 26 OctMachine LearningComputer Vision and Pattern Recognition

The gist

Sometimes, when computers teach themselves to recognize images, parts of what they learned can suddenly get worse without showing up in usual checks. The authors trained a system on fake images where they knew exactly what features mattered and found that even though the quality dropped drastically, the signals people normally use to watch training didn’t notice. Most checks and losses kept improving even when the system was losing knowledge. This means current ways to watch self-learning might miss when the system starts to forget.

Open → 2610.35766v1

Routing signals improve accuracy detecting missing objects in vision models

From Routing Signals to Selective Review: Visual regrounding in MoE VLMs

Abstract: Vision-language models (VLMs) may accept false visual premises, answering questions about a target object's color, count, location, or state even when it is absent. We call this reliability-critical behavior a target-absence grounding failure. Existing visual-grounding detectors primarily rely on generated responses, hidden states, or uncertainty measures. We present the first framework to leverage internal routing decisions in Mixture-of-Experts (MoE) VLMs to detect target absence before generation and guide selective correction. We extract target-token routing probabilities from Qwen3-VL-30B-A3B-Instruct and Gemma-4-26B-A4B-it, train a separate L2-regularized linear detector for each model, and use its predictions to selectively invoke a target-aware review prompt. Using routing alone, the Qwen and Gemma detectors achieve ROC-AUCs of 0.9988 and 0.9956 on GQA-Inpaint and retain 0.8095 and 0.7781 on the external OBER dataset, respectively. The resulting routing-gated policy improves end-to-end accuracy on GQA-Inpaint and OBER by +22.25% and +12.17% for Qwen, and by +13.42% and +1.39% for Gemma, without modifying model weights. Further analysis shows that the signal is localized to the target-object token, emerges in early MoE layers, and is distributed across partially substitutable experts. Although cross-dataset threshold shifts require recalibration, false-positive review causes limited harm overall, suggesting that intervention risk can be controlled through joint selection of the detector threshold and review prompt. Overall, we show that routing probabilities alone preserve actionable information about visual perception, allowing computation already produced by an MoE VLM to support low-cost detection and selective visual regrounding.

Tue 29 SeptComputation and LanguageComputer Vision and Pattern Recognition

The gist

Vision-language models sometimes confidently answer questions about objects that aren't actually in a picture, which can cause mistakes. The authors found that by looking inside the model at how it routes information between its parts, they can detect when an object is missing before the model even answers. They trained simple detectors on these routing patterns that identify failures very accurately, allowing the model to ask for a second check only when needed. This method improved accuracy significantly without changing the model itself and kept errors low when wrongly triggering rechecks.

Open → 2609.38111v1