Summary
Handling large, messy collections of evidence is tough for government intelligence because secure, cost-effective language models don’t offer lots of tools or ways to check their work. The authors created CLEO, a way to carefully add just the right tools and rules around a self-hosted language model so it works better without becoming complex or hard to audit. With CLEO, the system performed almost as well as a much bigger, more complex model but stayed simpler and easier to control.
Authors
Shai Nahum Gefen, Vered Chassida, Dmitry Basin, Guy Feigenblat
Abstract
Government intelligence teams need LLM agents that can analyze large, messy evidence collections, but security and cost constraints often rule out hosted frontier models and broad, hard-to-audit tool surfaces. We report CLEO: Constrained Loop for Environment Optimization, a method for fitting a minimal agent harness to a fixed, self-hostable LLM. Given an expert-validated dataset, CLEO admits only the tools, guardrails, memory policies, and loop components that improve accuracy while remaining general and auditable. On our held-out dataset, the CLEO harness reaches an 83.8% pass rate, within 1.2 points of the strongest out-of-the-box Claude Code SDK baseline, while being two orders of magnitude smaller.