SearchJev speeds up search agents with better calibrated fast decisions

SearchJev: A Fast and Calibrated System-1 Model for Search Agents

Information RetrievalComputation and Language

Summary

Search agents help computers decide quickly what information is relevant and what to do next. The authors built SearchJev, a fast decision-making system that separates quick choices from deeper thinking to work faster and more reliably. SearchJev uses a new way to learn and measure confidence in decisions, making it better at knowing when it should ask for help. Their tests show it makes decisions much faster and with better accuracy, improving overall search performance.

What this means in practice

Authors

Congfeng Cao, Lipeng Zuo, Konstantinos Papakostas, Qiwei Xu, Songwei Xu, Lun Zhou, Zhaochun Ren, Yougang Lyu, Xiaohui Yan

Abstract

Search agents repeatedly make short decisions about relevance, evidence sufficiency, and search actions. Using generative language models for these decisions introduces latency and unreliable confidence. We present SearchJev, a fast and calibrated System-1 model that separates search decisions from System-2 reasoning and generation. Given a search state and a decision schema, SearchJev directly scores legal options without autoregressive output generation. We propose Soft-Label Learning for Calibrated Decisions (SLCD) to learn decision probabilities from uncertain supervision and calibrate their confidence. In a dual-system search agent, SearchJev handles short decisions and delegates uncertain judgments to System 2, which retains planning, query generation, and answer composition. We also introduce SearchDecision-Bench, a benchmark unifying six types of search decisions for training and evaluation. On SearchDecision-Bench, SEARCHJEV improves decision quality over same-size Qwen3.5 autoregressive models, achieves 5.2-5.3 times faster decisions, and reduces average expected calibration error by 41-74%. On BrowseComp-Plus, the dual-system agents achieve a 3.7-4.7 times speedup in active search time while improving answer accuracy from 45% to up to 54%.