Code language models improve handling of noisy user instructions
Noisy Test-Time Reinforcement Learning for Code LLMs
Artificial Intelligence
Summary
Code language models usually work well with clear, error-free instructions, but real users often provide vague or mistaken requests. The authors created a method that helps these models improve themselves during testing by learning from messy, unlabeled examples without needing clean training pairs. Their technique cleans up noisy code suggestions conservatively and combines multiple program versions to guide better output. Tests show this approach makes code models more reliable even with different types of noise in user input.
What this means in practice
- •For software development teams: Improve code generation tools to handle imperfect and vague user requests more reliably without extra labeled data.
- •For devops engineers: Enhance automated code testing and deployment systems by stabilizing outputs from code language models under noisy input conditions.
Authors
Xikai Yang, Hieu Trung Nguyen, Dunyuan Xu, Yuzhi Zhao, Jinpeng Li, Wenao Ma, Pheng-Ann Heng
Abstract
Large language models (LLMs) have demonstrated remarkable performance across various code-related tasks. However, unlike carefully curated datasets that are typically high-quality and error-free, real-world user instructions are often vague and error-prone, posing significant challenges to the robustness of code LLMs. Furthermore, robustness-oriented fine-tuning relies on paired clean-noisy samples, which are costly to curate and require sophisticated noisy simulation techniques. To address these challenges, we propose the Noisy Test-time Reinforcement Learning framework (NTRL-Code), which enables robust self-evolution of code LLMs using only unlabeled noisy data during the testing stage. Specifically, NTRL-Code uses conservative self-denoising to obtain a cleaner semantic anchor for target estimation, and employs an abstract-syntax-tree (AST)-based structural aggregation mechanism to estimate a proxy target from multiple candidate programs. The policy is then optimized on the original noisy prompts with a hybrid reward that combines format validity, code similarity, and anti-repetition signals. Extensive experiments on three benchmarks, each incorporating character-level, word-level, and paragraph-level perturbations, demonstrate that NTRL-Code yields robust and consistent improvements, stabilizing the predictions of various base models. Our code is available at https://github.com/Xikai97/NTRL-Code.