Llms improve continuously with environment-driven dynamic policies
Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement
Computation and Language
Summary
Large language models often struggle to keep improving as their tasks and environments change. The authors propose a method called Dynamic Retrieval-based Policy Generation (DRPG) that helps these models learn smarter by creating specific plans based on past experiences and environment feedback. This approach makes the models better at a variety of tasks, like answering questions or writing code, without repeating the same mistakes. The authors also found that DRPG works well with different kinds of models and tasks, making it a flexible way to keep improving language models over time.
What this means in practice
- •For software development teams: Improve code generation models to adapt continually to evolving programming tasks and environments.
- •For medical data analysts: Enhance medical diagnosis models by dynamically generating policies that improve accuracy using past case data and feedback.
Authors
Ting-Wei Chang, Po-Chun Chen, Hen-Hsen Huang, Hsin-Hsi Chen
Abstract
Large Language Models (LLMs) have achieved remarkable progress across diverse domains, but continual adaptation to evolving tasks and environments remains a key challenge. Existing memory-augmented approaches retrieve individual past examples as direct references, but do not explicitly synthesize actionable strategies from them, causing the same types of errors to recur. We propose Dynamic Retrieval-based Policy Generation (DRPG), a framework that integrates memory-based retrieval with a dynamic policy generator, leveraging historical data and environment feedback to produce task-specific policies for continual LLM improvement. We evaluate DRPG across six benchmarks spanning text-to-SQL, question answering, medical diagnosis, and Python programming, using seven LLMs from both proprietary and open-weight families. DRPG outperforms strong baselines across most datasets and models. Further analysis demonstrates that DRPG's policy generation is robust to retrieval strategy, operates effectively without prior policy continuity, and can leverage smaller or cross-family models as cost-efficient policy generators. We also find that the benefit of policy-level guidance depends on task characteristics, offering practical insights into when and under what conditions this mechanism is most effective.