Reinforcement learning improves quantum neural network training speed and accuracy
Modeling quantum neural network gradient with reinforcement learning
Emerging TechnologiesNeural and Evolutionary Computing
Summary
Training quantum neural networks is tricky because their gradients can vanish, making learning very slow, and calculating these gradients becomes extremely demanding as networks grow. The authors developed RLQ-Grad, which uses a classical reinforcement learning agent to suggest updates for quantum neural network parameters instead of calculating gradients directly. This approach avoids problems with vanishing gradients, reduces memory and time costs drastically, and improves accuracy on several tests. It works well on quantum circuits up to 20 qubits, running thousands of times faster than traditional methods while maintaining good performance.
What this means in practice
- •For quantum algorithm developers: Train quantum neural networks more efficiently on near-term quantum hardware by using reinforcement learning-based parameter updates.
- •For quantum hardware engineers: Reduce computational and memory demands during quantum circuit optimization, enabling larger quantum system experiments.
Authors
Nhan Trong Luu, Duong Trung Luu, Nam Ngoc Pham, Thang Cong Truong
Abstract
Training quantum neural networks (QNNs) on near-term hardware remains hampered by two compounding difficulties: the exponential vanishing of gradient variance known as the barren plateau, and the $\mathcal{O}(L \cdot 2^n)$ time and memory cost of differentiating through an $n$-qubit, $L$-layer circuit. We propose RLQ-Grad, a reinforcement-learning-based optimizer in which a classical policy $π_φ$ (a spectrally-normalized PPO agent) learns to propose parameter updates directly, conditioned on the QNN's current parameters, loss, accuracy, and previous update. Because the surrogate gradient is emitted by a classical network rather than obtained by differentiating through the unitary $U(θ)$, its variance is not constrained by the barren plateau concentration bound, and its cost scales with the number of trainable parameters rather than the Hilbert-space dimension. We prove these properties formally and verify them on a hardware-efficient ansatz across four supervised benchmarks with up to $n=20$ qubits. RLQ-Grad preserves a near-flat gradient-variance curve where backpropagation, parameter-shift, and adjoint differentiation decay by 1 to 2 orders of magnitude. Accounting for the full training pipeline (PPO rollouts, actor-critic updates, and optimizer states), RLQ-Grad needs under 2 MB of memory and runs $2490\times$, $7876\times$, and $673\times$ faster per iteration than these three methods at $n=20$. It improves top-1 accuracy by up to $+10\%$ over gradient-based baselines on circuits of up to 12 qubits, and matches dedicated barren plateau mitigation methods on CIFAR-10 at 14 to 20 qubits, where evolutionary and gradient-free optimizers collapse to chance.