Reinforcement learning builds better error-correcting codes quickly
ZeroCode: On-demand Error-Correcting Code Construction from the Zero Matrix via Reinforcement Learning
Machine Learning
Summary
Error-correcting codes help fix mistakes in digital data, but making codes to fit different needs is hard. The authors developed ZeroCode, a computer program using reinforcement learning that builds these codes step-by-step from scratch. ZeroCode improves error fixing compared to earlier methods and allows for flexible rules, like avoiding certain patterns or keeping complexity low. It can generate many code options without needing to start over each time, making it easier to find good codes for various uses.
What this means in practice
- •For communications engineers: Design error-correcting codes customized for specific wireless or storage systems using on-demand constraints.
- •For quantum computing developers: Create parity-check matrices that accommodate quantum error correction requirements with flexible structure constraints.
Authors
Ju-Hyeong Lee, Yongjune Kim, Sang-Hyo Kim, Dae-Young Yun, Hee-Youl Kwak
Abstract
Error-correcting codes (ECCs) are essential across diverse applications, from wireless communications and storage to quantum computing, yet each application imposes distinct design requirements on the parity-check matrix (PCM). To address these on-demand requirements in a unified framework, we propose ZeroCode, a reinforcement learning (RL)-based approach that constructs PCMs sequentially from the all-zero matrix. ZeroCode formulates construction as a discrete sequential decision-making problem and uses proximal policy optimization with action masking to select valid edges. ZeroCode achieves a gain of approximately 1 dB over the prior RL-based construction method at a bit error rate (BER) of $10^{-4}$ for the (32,16) code and outperforms existing genetic, differentiable, and classical code-design methods in our experiments. Beyond optimizing decoding performance, the masking mechanism allows on-demand structural constraints, such as a maximum degree, 4-cycle-free structure, and quasi-cyclic structure, to be flexibly incorporated. Moreover, a single policy rollout yields a library of PCMs with varying edge counts, offering trade-offs between decoding performance and complexity without retraining. Overall, ZeroCode addresses diverse code-design requirements within a unified framework, providing solutions with optimized decoding performance under given constraints.