Multi-agent learning improves antenna placement and transmission in massive MIMO networks
Multi-Agent Reinforcement Learning for Movable Antenna-aided Cell-Free Massive MIMO Systems
Information Theory
Summary
Optimizing where to place movable antennas and how to transmit signals in massive wireless networks is very hard because the antennas have rules about how close they can be to each other. The authors developed a new learning method that splits the big problem into smaller coordinated ones, letting different agents learn how to adjust antenna positions and transmissions effectively. They also included special techniques to handle geometric constraints and to help the agents work together efficiently. Their approach improved network performance significantly in tests and needed less communication between agents compared to fully centralized methods.
What this means in practice
- •For wireless network operators: Improve the coordination and placement of movable antennas in large-scale networks to boost data rates with less overhead.
- •For cellular infrastructure engineers: Deploy adaptive antenna positioning and transmission strategies that efficiently handle complex constraints in massive MIMO systems.
Authors
Bokai Xu, Jiayi Zhang, Shuaifei Chen, Ziheng Liu, Huahua Xiao, Derrick Wing Kwan Ng, Bo Ai
Abstract
The inherent non-convex minimum-separation constraints introduced by movable antennas present a formidable challenge to the joint optimization of antenna positions and transmission strategies, rendering conventional methods computationally infeasible, particularly in large-scale cell-free massive multiple-input multiple-output (MIMO). In this work, we propose the graph-based learning individual intrinsic reward heterogeneous-agent proximal policy optimization (GLIIR-HAPPO) algorithm, a novel heterogeneous multi-agent reinforcement learning (MARL) framework that fundamentally overcomes this impasse by systematically decomposing the original coupled optimization into coordinated subproblems. To ensure tractability, we embed the non-convex geometric constraints into a penalty-augmented reward structure and develop a specialized geometric solver that enables the positioning agents to efficiently navigate the high-dimensional action space. Specifically, we propose an architecture featuring a dynamic-interaction graph critic for adaptive cross-role coordination, together with role-conditioned federated distillation that synchronizes same-role policies through compact actor-output statistics. Beyond architectural design, we establish a rigorous theoretical analysis that derives monotonic performance improvement bounds and establishes convergence guarantees for the proposed bi-level optimization. Numerical simulations demonstrate that our framework yields significant sum-rate improvements over state-of-the-art MARL schemes. Moreover, the performance of our advanced architecture closely approaches its fully centralized counterpart, while drastically reducing communication overhead.