Small local models improve accuracy with deterministic retrieval and context hydration
Noēsis: Deterministic-First Retrieval with Two-Tier Context Hydration for Factuality-Critical Queries on Small Local Models
Information RetrievalArtificial IntelligenceComputation and Language
Summary
Getting exact numbers right is very important in fields like healthcare, finance, and media, where wrong facts can cause real harm. The authors show that small language models often make up plausible but false numbers even when the right information is available. They propose Noēsis, a system that carefully checks facts deterministically before answering, using a special two-step method for adding context. This approach helps small models give precise and trustworthy answers quickly while always showing exactly where each fact comes from.
What this means in practice
- •For healthcare data teams: Provide exact dosage and lab value information from local models with guaranteed traceability and zero fabricated numbers.
- •For financial risk analysts: Retrieve and report precise figures and citations from limited local models ensuring every value is verifiable and no confabulation occurs.
Authors
Nicola Cogotti
Abstract
A wrong number is worse than no answer. Across factuality-critical domains -- audience metrics, scheduling and rights in media; dosages and lab values in healthcare; figures and citations in finance and legal -- a confident but fabricated value is more damaging than an honest admission of uncertainty. Yet this is the dominant failure mode we observe on small local language models: even when correct evidence is present in context, models fabricate plausible numbers and timestamps. Recent work characterizes a real limit of this regime: below 7B parameters, the bottleneck of retrieval-augmented generation (RAG) is not retrieval quality but context utilization. We present Noesis, the deterministic-first query plane of the Noesis architecture, which makes every deterministic judgment before generation. Its mechanisms follow from the ingestion architecture (subject of a separate patent application): (a) a producer-side fact layer rendering precomputed metric facts verbatim without ranking; (b) positional addressing with deterministic cross-source alignment, resolved ahead of query time at zero LLM cost; (c) provenance scoping as an attribution constraint with multi-tier named-reference routing; and (d) two-tier context with model-triggered verbatim hydration. Across four ablations, a 2B model reaches parity with a 35B model on factual integrity (exact values in all runs; zero confabulated numbers on absent-entity traps); structured retrieval beats flat RAG by +11.4 points at 2B; skeleton-only context preserves quantitative answers at 20-30% smaller prompts; and hydration recovers verbatim narrative in ~8s versus ~29s. Two properties matter for regulated domains: each query resolves in a single generation call, and every reported value is traceable to its exact source and position by construction.