AI summaryⓘ
The authors study how to manage connections and energy in busy wireless networks fairly and efficiently, considering strict limits on energy use and switching between base stations. They note that standard multi-agent reinforcement learning struggles because it can't check constraints every moment or easily break down the fairness goal by time. To fix this, they introduce HeLyMARL, a method that uses virtual queues to balance energy and switching limits in real-time within a unified reward system. Their experiments show HeLyMARL keeps the network fair and efficient without running out of energy or switching budgets prematurely, outperforming other methods. This helps maintain steady service in dense wireless setups.
Proportional fairnessMulti-agent reinforcement learningBase station activationHandover controlEnergy constraintsLyapunov optimizationVirtual queuesThroughput maximizationFinite-horizon constraintsScheduling
Authors
Yeonseo Jeong, Wonhyeok Ko, Sungweon Hong, Songnam Hong
Abstract
Maximizing throughput under proportional fairness in dense wireless networks requires jointly managing user association, scheduling, base station (BS) activation, and handover control under hard finite-horizon energy and handover budgets, which induces a fundamental tension between BS-side energy management and user-side handover regulation. While multi-agent reinforcement learning (MARL) is a natural framework for such distributed sequential control, its application here faces two difficulties: finite-horizon budget constraints cannot be evaluated at each time slot, and the nonlinear proportional fairness utility admits no principled per-slot decomposition. We propose HeLyMARL, a Lyapunov-embedded heterogeneous MARL framework that resolves both via drift-plus-penalty decomposition with virtual queues. The energy and handover constraint pressures are internalized directly into a unified per-slot reward, converting the constrained finite-horizon problem into an unconstrained MARL problem. Comparison against two Lagrangian-based alternatives reveals a timescale separation: Lagrangian relaxation regulates constraints only across training episodes, whereas the virtual queues of HeLyMARL bound cumulative budget consumption at every partial horizon within an episode, a pacing guarantee beyond the reach of greedy Lyapunov-based control. Simulations show that HeLyMARL is the only method that sustains the throughput-fairness balance together with uninterrupted service throughout the horizon, outperforming conventional MARL, Lyapunov-based, and constrained MARL benchmarks without premature budget exhaustion.