Llm debugging agents struggle without live database access
Where LLM Agents Get Stuck Debugging Databases: Access More Than Reasoning
Software Engineering
Summary
When large language model (LLM) agents try to fix database problems that happen when many things happen at once, they often guess wrong because they can’t see enough live system information. The authors studied one such agent that kept making wrong fixes and then undoing them, especially for problems caused by complex timing and locking issues. They found that the agent’s limited access to real-time data like locks and deadlock details was the main barrier. The authors suggest giving these agents special safe access to detailed system logs and live views to help them diagnose better.
What this means in practice
- •For production database teams: Use enhanced data access layers to help automated agents diagnose and fix concurrency-related incidents in live databases more accurately.
- •For devops engineers: Integrate safe interrogation APIs and replay systems to provide debugging agents richer live context for runtime faults in distributed systems.
Tested on one dataset.
Authors
Gustavo Pinto
Abstract
Large language model (LLM) coding agents are increasingly asked not only to write code but to debug running systems from live runtime data. We report a field observation of one such agent (Claude Code) as it diagnosed eight database incidents in a production message-dispatch engine. In every incident whose root cause depended on runtime concurrency, the agent announced a cause, shipped a fix, and later walked it back, and each wrong guess cost a deploy-and-observe cycle. Mapping each incident to what the agent lacked, we find that access to production state was the most pervasive bottleneck. The missing access included live lock views, untruncated deadlock reports, read-only execution, and an explicit environment context. An access gap was present in all five concurrency-dependent incidents: alone in one, and in four combined with a reasoning error such as anchoring on an early cause, incomplete exploration of the configuration, or a misreading of the code's transaction semantics. The three incidents that did not depend on concurrency needed nothing beyond what the agent already had. Within the incidents, the agent's wrong hypotheses were consistent with the evidence it could see; they were not fabrications. We argue that the next step for production-debugging agents is a forensics layer that gives the agent the access it lacks inside existing security controls. Such a layer would combine a safe interrogation API, a lock-and-deadlock flight recorder, and a production replay harness.