History-aware reinforcement learning improves dense layout routing efficiency
Routing Dense Layouts with History-Aware Offline Reinforcement Learning using LSTM
Hardware ArchitectureMachine Learning
Summary
Routing tiny wires on computer chips is becoming very hard because circuit designs are getting denser and have more rules to follow. The authors found that current learning methods struggle with these dense layouts. They developed a new approach that remembers previous routing attempts using a small LSTM memory, helping the system learn better cost settings during routing. This significantly reduces errors and speeds up the process without changing the main routing algorithm.
What this means in practice
- •For chip layout engineers: Improve detailed routing convergence and reduce design errors in complex, dense chip designs using a history-aware offline RL policy integrated into existing routing tools.
- •For eda tool developers: Incorporate lightweight LSTM-based cost prediction into routers to speed up runtime and improve accuracy on challenging placement and routing scenarios.
Authors
Afsara Khan, Austin Rovinski
Abstract
Detailed routing remains a dominant runtime bottleneck in physical design due to increasing complexity of design rules. Modern routers can struggle to resolve persistent violations under dense operating conditions. While recent work leverages reinforcement learning (RL) to dynamically select costs for each routing iteration, we find that this technique struggles with high-density designs where routing solutions are significantly harder. To address this, we present a history-aware offline RL policy which predicts iterative cost weights in these dense regimes to improve convergence across placement densities by utilizing readily available features from the router. Our policy uses conservative Q-learning similarly to prior work; however, our key insight is that addition of a lightweight LSTM architecture and additional features can retain sequence context and improve routing convergence across multiple densities and route guide qualities. Our policy can be integrated into any cost-based router with minimal pipeline changes, as it does not interfere with the core search algorithm. We evaluate our policy on held-out density and adjustment settings, including difficult operating points induced by dense placement and low guide quality. Our policy reduces design rule violations (DRVs) by an average of 92% over the top public baseline while simultaneously reducing runtime by 10%.