History aware reinforcement learning speeds up dense circuit layout routing
Routing Dense Layouts with History-Aware Offline Reinforcement Learning using LSTM
Hardware ArchitectureMachine Learning
Summary
Routing tiny wires on computer chips is getting more complicated because chip designs are very dense, causing delays and mistakes. The authors found that current learning methods struggle with these dense designs. They developed a new technique that remembers past routing steps using a memory-based model called an LSTM to better guide the routing process. This approach significantly reduces design errors and speeds up routing, making it easier to handle complex chip layouts without changing existing routing methods much.
detailed routingphysical designdesign rule violationsreinforcement learningoffline RLLSTMQ-learningcost weightschip layoutroute guide
Authors
Afsara Khan, Austin Rovinski
Abstract
Detailed routing remains a dominant runtime bottleneck in physical design due to increasing complexity of design rules. Modern routers can struggle to resolve persistent violations under dense operating conditions. While recent work leverages reinforcement learning (RL) to dynamically select costs for each routing iteration, we find that this technique struggles with high-density designs where routing solutions are significantly harder. To address this, we present a history-aware offline RL policy which predicts iterative cost weights in these dense regimes to improve convergence across placement densities by utilizing readily available features from the router. Our policy uses conservative Q-learning similarly to prior work; however, our key insight is that addition of a lightweight LSTM architecture and additional features can retain sequence context and improve routing convergence across multiple densities and route guide qualities. Our policy can be integrated into any cost-based router with minimal pipeline changes, as it does not interfere with the core search algorithm. We evaluate our policy on held-out density and adjustment settings, including difficult operating points induced by dense placement and low guide quality. Our policy reduces design rule violations (DRVs) by an average of 92% over the top public baseline while simultaneously reducing runtime by 10%.