Reflex architecture cuts large model calls by over 70 percent
REFLEX with Jev for Efficient Selective Control in LLM Agents
Artificial Intelligence
Summary
Making decisions with large language models can be slow and costly. The authors studied REFLEX, a system that quickly handles easy decisions with a simpler component called Jev and only asks the bigger model when unsure or more complex answers are needed. This approach keeps success rates high while reducing use of big models by nearly three-quarters. However, the benefits depend on the number of choices and the clarity of decision points, with limited gains if simpler routing is already very accurate.
What this means in practice
- •For software engineers: Reduce usage of large, compute-heavy language models in AI agents by integrating a faster decision layer to handle routine queries.
- •For cloud service providers: Lower operational costs in AI services by selectively calling strong language models only when simpler decisions cannot be made confidently.
Authors
Tiantong Wu, Wei Yang Bryan Lim
Abstract
LLM agents often use generative models for bounded decisions, raising the question of when these decisions can be handled more efficiently without reducing task success. We study REFLEX, an agent architecture that uses Jev as a fast, typed decision layer and calls a strong LLM when confidence is low, or generation is required. On a frozen 100-task benchmark, REFLEX achieves 95% success with 72.7% fewer strong-model calls than a strong-only agent, with reductions persisting across three fallback families. Controlled interventions show that reliability depends on action-set size and near-valid alternatives near authorization boundaries. External BFCL and $τ$-style evaluations reveal limited advantages over a cheap generative cascade when ordinary routing is already highly accurate. These findings identify when selective control with Jev can reduce computation and where its benefits are limited.