AI summaryⓘ
The authors study how changing tasks can cause some neurons in multi-agent reinforcement learning systems to stop working, which hurts learning. They point out that simply resetting inactive neurons can be risky because some appear inactive but are still important. Their method, called PRIME, carefully checks both neuron activity and training signals before resetting neurons that are truly inactive, helping the system keep useful knowledge while regaining learning ability. Tests on a drone communication simulation showed that PRIME improved performance and kept fewer neurons inactive. The authors also provide a theoretical guarantee that the cost of these resets depends on the smaller inactive part of the network, not the entire system size.
reinforcement learningmulti-agent systemsnon-stationarityneuronsactivationgradientpolicyparameter sharingresetting neuronsdynamic regret bound
Authors
Wen Qiu, Zhiqiang He, Wei Zhao, Hiroshi Masui
Abstract
Most reinforcement learning controllers for these networks assume stationary conditions, and the few that handle change react to the external environment while leaving the network's internal state unexamined. We show that sustained non-stationarity damages this internal state directly: as objectives shift, neurons progressively fall dormant and the shared policy loses the capacity to learn. The obvious remedy, resetting dormant neurons, is unsafe under shared-parameter multi-agent training: many neurons that appear inactive are still receiving strong training gradients, and whether a neuron appears dormant depends on which agent's observations it processes. PRIME (Plasticity Recovery In Multi-agent Environments) therefore verifies both directions before intervening. Extending the bidirectional Silent Neuron framework to cooperative multi-agent reinforcement learning, it aggregates activation and gradient statistics over the full team batch, reads the backward signal from the gradient the training loss has already deposited , not from a hand-crafted proxy, and reinitializes only neurons that are simultaneously activation-dormant and gradient-silent. Useful representations are preserved while learning capacity is restored. On a phase-switching UAV emergency communication simulator, PRIME improves interquartile mean return by 24.9\% over MAPPO and holds dormant neuron fractions at 10--20\% versus 40--45\%; ablations attribute the gains to the gradient signal and team-level aggregation rather than to the specific reset operator. A dynamic regret bound shows that the perturbation cost scales with the small silent-subspace dimension rather than the full parameter count.