Reinforcement learning improves random access preamble allocation for diverse traffic
QoS-Aware RACH Preamble Slicing via Quota-Projected Branching Deep Reinforcement Learning
Networking and Internet Architecture
Summary
Cellular networks let many devices connect by using limited 'preambles' to request access. Efficiently sharing these preambles is tricky when devices have very different needs, like urgent versus routine messages. The authors developed a smart system that learns how to split preambles to match these needs better, improving success rates and reducing delays. Their system uses a branch-based deep reinforcement learning method to decide allocation based on traffic type and urgency.
What this means in practice
- •For cellular network operators: Optimize resource allocation in random access channels to improve connection success and reduce delays for heterogeneous device traffic.
- •For wireless network equipment vendors: Incorporate advanced reinforcement learning controllers to better manage preamble assignments across different access procedures in base stations.$Commercial implications: Enables product differentiation with intelligent resource management in cellular infrastructure hardware.
Authors
Jiulin Guo, Jiahan Xu, Jiashuo Zhang, Heng Yang, Yizhen Sun, Yutong Xie, Shanshan Li, Zhenyu Liu, Lei Zhang
Abstract
Quality-of-service (QoS)-aware random access requires adaptive allocation of a finite random access channel (RACH) preamble budget across heterogeneous traffic and access procedures. This paper proposes QP-BD3QN-RACH, a quota-projected branching deep reinforcement learning controller for mixed two-step (2RA) and four-step (4RA) contention-based random access. Four action branches correspond to the delay-sensitive and delay-tolerant 2RA/4RA preamble pools. A branching dueling Double DQN selects pool-specific multipliers, and deterministic quota projection converts them to nonnegative integer allocations that preserve the preamble budget. With five actions per branch, the controller represents 625 pre-projection branch-action tuples using 20 branch-action outputs. Evaluation covers five arrival loads, cross-method comparison under nominal seed 42, six-seed sensitivity of QP-BD3QN-RACH, and targeted ablations. Across the five-load grid, its mean direction-aligned differences relative to four comparators are positive: 5.74 to 8.21 percentage points for success/collision, 1.23 to 1.92 percentage points for fallback, 0.35 to 0.68 percentage points for blocking, and 0.128 to 0.456 decision intervals for successful-access delay. Load-wise results exhibit metric-dependent tradeoffs, particularly under intermediate and overload conditions.