WeEnv cuts environment setup time for agentic reinforcement learning
WeEnv: The Environment for Agentic Reinforcement Learning at WeChat
Distributed, Parallel, and Cluster Computing
Summary
Agentic reinforcement learning runs tasks inside complicated environments like virtual machines, but setting up these takes a lot of time and slows down learning. The authors found this happens because managing these environments through their entire lifecycle is hard. They created WeEnv, a system that manages packaging, starting, and adjusting environment resources efficiently. This system makes environments start faster and use resources more flexibly, greatly reducing wasted time during learning iterations.
What this means in practice
- •For machine learning platform engineers: Manage and speed up environment setup and resource use for agentic RL tasks inside virtual machines or containers.
- •For cloud infrastructure teams: Provide elastic CPU and memory quotas to optimize resource usage for tasks with varying demands in complex environments.
Authors
Yang Yu, Jing Lei, Shaoxun Zeng, Xinyu Gao, Jindi Shi, Ci Lei, Junjie Zhang
Abstract
Agentic reinforcement learning (RL) differs from conventional RL in that every task executes inside a complex environment, e.g., a virtual machine or a container. We find that agentic RL pays a heavy environment tax: a large share of the iteration time goes to the environment rather than to learning. The root cause is the lack of a full-lifecycle solution to environment management. We present WeEnv, which manages environments across packaging, initialization, and provisioning. WeEnv packages components as independently published layer groups and composes them at initialization, so that updating a component republishes one small group rather than every artifact containing it. To speed up environment initialization, WeEnv launches environments instantly and fetches contents on demand. During task execution, WeEnv provisions CPU and memory elastically, adjusting each environment's quota from its observed usage to fit the varying demands. WeEnv reduces the initialization by 5.6-14.2x over E2B, Docker, and AgentENV, cutting its share of the iteration time from up to 53.4% to 9.1%. WeEnv is deployed for agentic RL at WeChat.