Summary
The paper studies general decision models, which are faster alternatives to large language models for making structured decisions. The authors find these models work well when decisions can be clearly made from given information but struggle when expert knowledge or careful handling of uncertainty is needed. They also show that making quick decisions locally can cause more errors when chained over many steps in large systems. However, in social simulations, decision models can predict individual responses nearly as well as big language models but with some biases and larger errors overall. The authors introduce new hybrid models that combine fast decisions with reasoning, achieving similar performance to existing models but at much faster speeds.
What this means in practice
- •For software engineers: Implement faster decision-making components in dynamic multi-step systems while being aware of error accumulation trade-offs.
- •For simulation developers: Use general decision models to reduce computational cost in social simulations while maintaining reasonable accuracy for individual response prediction.
Authors
Feiyu Duan, Jiayu Lin, Jia Wang, Jun Xiang, Jialiang Wu, Xinnong Zhang, Hanqi Yan, Siyuan Wang, Zhongyu Wei
Abstract
General decision models, such as Jev, have recently emerged as efficient alternatives to LLMs for structured judgment and selection. But what kinds of decisions can these models reliably make, and how does their behavior change when individual decisions are composed into larger systems? To study this, we introduce JEVal, a bilingual benchmark comprising 11,257 instances from 36 datasets across 10 application domains, and evaluate 25 model configurations spanning general decision models and generative LLMs. Our results show that (1) general decision models are most competitive when decisions can be resolved from available evidence, but weaken when they require specialist knowledge or faithful uncertainty estimation: they can often identify the most likely outcome while substantially overstating its probability. (2) In more dynamic and realistic systems involving long-horizon, multi-step interactions, the advantages of fast local decision making are offset by reliability failures at the system level. on $τ$-bench, faster local decisions reduce median episode time but lower task success as decision errors accumulate over long trajectories. (3) In large-scale social simulation, decision models approach strong generative LLMs on individual response prediction at substantially lower inference cost, yet remain weaker in user profiling and exhibit larger aggregate estimation errors and systematic bias. Finally, we propose InnerJev-4B and InnerJev-27B, which internalize an open-weight LLM's own reasoning into a single-pass first-token decision through Reasoning-to-Readout Self-Distillation, with InnerJev-27B performing on par with Jev on JEVal while answering a typical query in about 0.1 s.