Self-correction in coding agents improves program fixes with adaptive evidence weighting
How Much Evidence Should a Coding Agent's Self-Correction Carry? Adaptive Dirichlet Evidence for Self-Distillation
Artificial IntelligenceSoftware Engineering
Summary
Coding agents that write or fix programs learn better when they weigh their own mistakes correctly. The authors created a method called Effective-Evidence Self-Distillation that helps these agents figure out how much to trust the evidence from their past corrections. This method treats how relevant the feedback is and how much information supports it as separate things, improving the learning process. When tested, this approach helped models write more accurate code and improved their success rate on coding tests.
What this means in practice
- •For software development teams: Improve automated code generation tools by enabling them to better learn from their own program corrections for more accurate fixes.
- •For automated debugging teams: Enhance debugging systems that use self-correction signals to reduce future coding errors by adapting learning weights based on evidence reliability.
Authors
Yunbo Long, Guangya Hao, Yuhan Liu, Yiting Duan, Longyan Tan, Yunchen Long, Hao Wu
Abstract
Execution feedback lets coding agents revise programs and learn from their own corrections. A correction's learning weight should reflect both the transitions supported by its executions and the amount of evidence behind that support. We introduce Effective-Evidence Self-Distillation (EESD), which represents these quantities separately. Normalized execution relevance determines relative transition support and an effective pseudo-count mass; a Dirichlet posterior then produces an uncertainty-penalized weight for KL-anchored correction learning. Under a symmetric prior, changing mass preserves category ordering, and effective mass yields a supervised coefficient bounded by its matched fixed-mass counterpart. Across four model-domain history sweeps, increasing visible observations from one to eight reduces future-outcome NLL by 55.0-59.3%. At eight observations, effective mass achieves lower NLL than fixed mass in all four comparisons. In the primary matched DeepSeek/RunBugRun study, argmax predictions agree on all 3,000 examples, with the largest NLL gain under concentrated relevance. After one correction-learning round, DeepSeek/CodeARC all-tests Pass@1 increases from 15.0% to 20.4%, with a paired 95% source-bootstrap interval of [+2.8, +8.0] percentage points. The twelve-setting downstream evaluation establishes the model-domain scope of this update. These results show how separating evidence support from evidence mass changes probability estimation and correction learning in coding agents.