Closed loop system greatly improves meeting text output targets
A Closed-Loop Control Architecture for Reliable Constraint Satisfaction in LLM Text Generation
Software Engineering
Summary
When using large language models to create text that meets specific numeric rules—like word counts or readability scores—one attempt often doesn’t get it right. The authors show a system that generates text, checks if it fits the rules, then edits it repeatedly until it does. This looping method made the output meet targets much more reliably and kept important details intact. The system works by using code to decide if edits are good enough, rather than relying on the language model’s guesswork alone.
What this means in practice
- •For software developers: Create applications that generate text meeting exact numeric criteria like length or readability by automating editing and evaluation steps.
- •For content management teams: Ensure generated documentation or marketing materials adhere precisely to content length and quality standards using iterative model-guided revisions.
Authors
Quan Zhou, Shahbaz Siddeeq, Mika Saari, Pekka Abrahamsson
Abstract
Software systems increasingly embed a large language model in features that must satisfy a numeric output constraint, that is, a requirement expressible as a number or an interval and checkable by code, such as a target word count or a target readability grade band. Because such a model is non-deterministic, is configured through natural-language instructions rather than a typed interface, and satisfies a stated requirement only approximately, a single prompt neither reliably meets the target nor preserves the source content. This paper presents and evaluates a closed-loop control architecture for this problem. It has five stages: generate, evaluate, adjust, archive, and analyze. The model is called only to write and to edit text, while deterministic code compares a composite readability value against a target band, rejects any edit that drops source entities, numbers, or keywords, and makes every accept decision. Over 114 single-shot generation jobs and 240 closed-loop runs on four commercial models, single-shot prompting met the target in 21.1 to 31.6 percent of cases and the closed loop in 92.5 to 98.8 percent, within two edit rounds on average and at a recall-based fidelity of 0.92 to 0.93; the two models common to both settings show the same effect. Because the controller optimizes the value on which success is scored, the result establishes reproducible control over a declared, computable metric and not validated human difficulty. The transferable practice is to declare the acceptance condition as code, bound the model to local edits, and gate every edit on a content check.