Iris cuts latency in visual decision tasks with reusable cached options
Seeing the Options: Typed Decisions over Visual Candidates with Reusable Option Memory
Artificial IntelligenceComputer Vision and Pattern Recognition
Summary
Choosing the right item from many pictures or text options is tricky for AI systems, especially when the order of choices changes. The authors created Iris, a system that can handle images, text, or both as choices, while keeping decision-making fast and consistent no matter how options are ordered. Iris reuses memory from previous decisions to speed up future ones by a large margin and keeps accuracy high even when options are shuffled. This makes it easier and quicker for AI to pick the right choice in tasks like selecting user interface elements or recommending products.
What this means in practice
- •For ui developers: Speed up selection of interface elements with consistent performance despite changing option orders by using cached visual candidate encoding.
- •For ecommerce product teams: Deliver faster and stable product recommendations by reusing encoded image options from previous queries, reducing response time dramatically.
Authors
Fengxin Li
Abstract
Typed decision-making tasks in agents, such as UI element selection, product recommendation, and chart matching, require models to choose from a set of discrete candidates. While prior work demonstrates that text-only System One models achieve four key properties including candidate-order invariance, scalability, and cache reuse via separate request-candidate encoding, extending them to multimodal settings where candidates are visual remains underexplored. We introduce Iris, a framework that accommodates image, text, or hybrid candidates while preserving the core decision module. We prove that order invariance and exact cache reuse hold for any per-candidate encoder under Iris, and derive the closed-form latency reduction. We construct a visual-candidate benchmark from Multimodal-Mind2Web (K=16, deduplicated, order-shuffled, three generalization splits). Independent encoding (Iris) achieves 78.4 / 72.1 / 74.8 percent accuracy with zero-flip rate under candidate reordering. Joint encoding reaches 90.6 / 89.5 / 91.6 percent but suffers an 8 to 10 percent answer-flip rate when shuffled. To bridge this gap, we propose a design that encodes each candidate conditioned on the request state without inter-candidate cross-attention; it preserves strict order invariance, enables forked cache sharing, and attains 91.7 / 86.5 / 90.0 percent accuracy at one-third of OneJev latency with calibrated null probabilities (AUROC 0.93 to 0.97). Reusing image candidates across requests cuts latency by 37x at K=256 and 191x at K=2048. On MetaEncoder benchmarks, Iris-4B achieves 100 percent on JevBench original and easy, outperforming 30B MetaEncoder and 27B OpenJev. Finally, we resolve a systems bottleneck on Ampere GPUs: reformulating Qwen3.5 patch-embedding Conv3d into its exact matrix-product form accelerates processing by 350x. Code, data, and checkpoints: https://github.com/lllfx/Iris.