Small language models improve factual answers with clear source tracking
Noēsis: Deterministic-First Retrieval with Two-Tier Context Hydration for Factuality-Critical Queries on Small Local Models
Information RetrievalArtificial IntelligenceComputation and Language
Summary
Getting correct numbers is crucial in fields like healthcare and finance, but small AI models often make up answers even when the right information is available. The authors developed Noēsis, a method that carefully checks facts before generating answers, making sure all reported numbers come straight from trustworthy sources. This approach lets a small AI model perform as accurately as much bigger ones while keeping track of exactly where every fact came from. This is important because it helps users trust the answers, especially when mistakes could have serious consequences.
small language modelsfactualityretrieval-augmented generationcontext utilizationdeterministic retrievalprovenance trackingquery hydrationstructured retrievallocal AI modelsmodel parameter size
Authors
Nicola Cogotti
Abstract
A wrong number is worse than no answer. Across factuality-critical domains -- audience metrics, scheduling and rights in media; dosages and lab values in healthcare; figures and citations in finance and legal -- a confident but fabricated value is more damaging than an honest admission of uncertainty. Yet this is the dominant failure mode we observe on small local language models: even when correct evidence is present in context, models fabricate plausible numbers and timestamps. Recent work characterizes a real limit of this regime: below 7B parameters, the bottleneck of retrieval-augmented generation (RAG) is not retrieval quality but context utilization. We present Noesis, the deterministic-first query plane of the Noesis architecture, which makes every deterministic judgment before generation. Its mechanisms follow from the ingestion architecture (subject of a separate patent application): (a) a producer-side fact layer rendering precomputed metric facts verbatim without ranking; (b) positional addressing with deterministic cross-source alignment, resolved ahead of query time at zero LLM cost; (c) provenance scoping as an attribution constraint with multi-tier named-reference routing; and (d) two-tier context with model-triggered verbatim hydration. Across four ablations, a 2B model reaches parity with a 35B model on factual integrity (exact values in all runs; zero confabulated numbers on absent-entity traps); structured retrieval beats flat RAG by +11.4 points at 2B; skeleton-only context preserves quantitative answers at 20-30% smaller prompts; and hydration recovers verbatim narrative in ~8s versus ~29s. Two properties matter for regulated domains: each query resolves in a single generation call, and every reported value is traceable to its exact source and position by construction.