System 1 models enable fast and calibrated AI decision making
The Hitchhiker's Guide to System 1 Models: Fast Decision-Making Beyond LLMS
Artificial IntelligenceComputation and Language
Summary
Large Language Models can think carefully but are too slow for many quick decisions that AI systems need, like sorting or scoring things instantly. The authors describe a type of AI called System 1 models that make decisions very fast in a single step. They organize these models into five groups and explain how to train and use them effectively, focusing on how to measure and trust their confidence. Their guide also shows how these fast models can work together with slower models to balance speed and accuracy. They provide tools, examples, and checklists to help engineers build faster AI decision systems.
What this means in practice
- •For ai engineers: Build sub-second AI decision systems using fast System 1 models for tasks like classification and routing with better calibrated confidence.
- •For real-time system developers: Integrate System 1 and large language models to optimize workflow speed and accuracy by switching based on model confidence scores.
A survey. It maps existing work.
Authors
Haggai Roitman
Abstract
Large Language Models excel at reasoning and generation but their autoregressive, token-by-token inference makes them unnecessarily slow, expensive, and poorly calibrated for the many routine decisions (routing, gating, scoring, classification) that modern AI systems require at sub-second latency. A growing class of models addresses this gap: System 1 decision models, architectures that map an input directly to a typed, structured decision in a single forward pass. This guide provides a comprehensive, practitioner-oriented treatment of the field. We propose a five-family taxonomy covering encoder-based classifiers, reward models, non-autoregressive decision engines, contrastive decision models, and extraction models, and we provide in-depth coverage of each family's architecture, training methodology, and deployment trade-offs. A central theme is calibration: we develop the theory of proper scoring rules, analyze Reinforcement Learning for Calibrated Decisions (RLCD), and examine recently discovered phenomena including IIA violations, ordinal-scale bias, and few-shot recalibration under distribution shift. On the integration side, we describe how System 1 models serve as decision infrastructure within agentic workflows and System 1 + System 2 cascades that route between fast and slow models based on calibrated confidence. The guide covers 18 application domains, maps the rapidly growing open-source ecosystem (over 70 model reproductions in weeks), provides runnable code throughout, and includes a practitioner's checklist, an open-source model zoo, and 22 self-assessment questions with detailed answers. Spanning 190 references, it is intended as a unified entry point for researchers and engineers working at the intersection of fast decision-making, calibration, and production AI systems.