Adaptive agent learning improves decision making under changing rules
AdaptEvo: Adaptive Agent Learning with Evolving Supervision
Computation and Language
Summary
Making decisions based on written rules can be tricky because the rules might not cover every situation perfectly, and examples used to teach can be unclear or incomplete. The authors created AdaptEvo, a system that learns better by adjusting how much it trusts guidance and by improving itself when it notices repeated mistakes. It also updates how it checks decisions to catch overlooked errors. Their tests showed AdaptEvo works better than previous methods, especially when the rules change over time.
What this means in practice
- •For content moderation teams: Enhance automated content review by maintaining decision accuracy when moderation rules are updated or evolving.
- •For compliance monitoring teams: Support adaptive decision processes in regulatory audits where rules and evaluation criteria can change over time.
Authors
Shijun Wan, Jiancong Xie, Hang Xu, Jin Duan, Qixiong Wang, Xi Xiang, Maofei Que, Yahui Liu, Zhongyu Wei, Mu Chuan
Abstract
Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in decision guidance and process evaluation, while reference judgments vary in their support from the rules and evidence. To address these challenges, we introduce AdaptEvo, a framework for learning under imperfect supervision that couples confidence-adaptive policy optimization with evolving decision knowledge and evaluation rubrics. Its Training module uses Confidence-Adaptive GRPO (CA-GRPO) to balance outcome and process rewards according to reference confidence. Its Evolution module synthesizes reusable decision knowledge from recurring failures across training cases and refines process rubrics to detect overlooked errors. To support empirical evaluation, we construct an industrial multimodal content moderation dataset comprising a training set and In-Period and Out-of-Period test sets, with the latter collected under changed rules. Using Qwen3.6-35B-A3B, AdaptEvo achieves 61.9% exact-label accuracy and 72.2% binary decision accuracy on In-Period, exceeding GRPO by 7.5 and 3.7 percentage points, respectively. On Out-of-Period, the policy trained with CA-GRPO retains exact-label accuracy gains over the base model across evaluated checkpoints without injected decision knowledge, while GRPO declines with continued training. CA-GRPO also outperforms the tested fixed reward mixtures on both Out-of-Period metrics.