Self reflective loop improves evidence based text generation on edge devices
Self-Reflective Chain-of-Thought (SRCoT): Evidence-Based Text Generation via a Closed-Loop Retrieve--Reflect--Revise Pipeline on Edge LLMs
Artificial IntelligenceComputation and Language
Summary
Sometimes language models write answers that sound good but don’t clearly back up their facts or follow local rules well. The authors came up with Self-Reflective Chain-of-Thought (SRCoT), a step-by-step process where the system retrieves information, thinks about it, and revises its text to better reflect sources. It runs on a laptop and shows where information is missing or where evidence supports claims, making the outputs easier to check. The system aims to make text generation more transparent and controllable but is still a research prototype that needs human checking.
What this means in practice
- •For emergency response teams: Generate evidence-linked situational reports from local data on portable devices for enhanced decision-making during emergencies.
- •For software developers for edge ai: Implement modular text generation pipelines with explicit source tracking to improve transparency and control of language models on local hardware.
Tested on simulated data.
Authors
Fabian Broszio, Michael Hoffmann, Adrian Paschke
Abstract
Large language models (LLMs) can generate fluent outputs that lack source support or conflict with local operational constraints. We introduce Self-Reflective Chain-of-Thought (SRCoT), a modular architecture that organizes generation as a closed Retrieve--Reflect--Revise loop. SRCoT separates environment state, persistent knowledge, accumulated experience, and short-term memory into provenance-labeled stores. BM25 retrieval supplies local context; subsequent passes expose intermediate drafts, missing-information markers, follow-up queries, and optional claim--evidence mappings. These artifacts inform retrieval, revision, clarification, or abstention decisions. We examine the architecture through qualitative, trace-based case studies in two fully synthetic settings: emergency medical services and Munich-Sim, an emergency-operations simulation. Local prototype runs on an Apple M4-family laptop with 128 GB memory illustrate source-labeled documentation, explicit gap marking, and structured dispatch-style outputs. The traces also reveal provenance-assignment errors, incomplete evidence coverage, and unsafe assumptions that remain possible despite prompting constraints. These cases demonstrate report-level inspectability and local execution, not comparative superiority, hallucination reduction, clinical validity, or operational safety. The architecture separates retrieval and control from the model interface, but behavior across different models remains to be evaluated. SRCoT is a research proof of concept requiring human review, with its principal contribution lying in an inspectable control procedure rather than a new model.