Agentic inference boosts verification code quality and cost efficiency
Inference-Time Scaling in Agentic Hardware Verification Workflows: A Multi-Strategy Characterization Across Protocol Complexity
Hardware ArchitectureSoftware Engineering
Summary
Generating correct test environments for hardware verification is tricky because the code must work perfectly with complex protocols and compilers. The authors studied how increasing computing power during code generation—rather than before—affects quality and cost. They found that using structured feedback that targets specific types of errors is much more efficient than just relying on raw compiler messages. This better feedback helps find and fix subtle protocol bugs that look correct to the compiler but fail functionally. Their approach also uncovered hidden bugs in two test designs, showing real-world impact.
What this means in practice
- •For hardware verification engineers: Generate higher quality verification environments with fewer errors by using structured feedback during inference to guide testbench synthesis.
- •For electronic design automation tool developers: Integrate structured diagnostic feedback loops into testbench generation workflows to efficiently detect functional protocol faults and improve verification completeness.
Authors
Vikash Kumar
Abstract
Inference-time scaling, allocating more compute during generation rather than pre-training, has demonstrated dramatic quality improvements for LLM reasoning tasks. Its behaviour in structured hardware verification code generation, where outputs must satisfy compiler correctness, protocol semantics, and functional coverage constraints simultaneously, is less understood. We present a systematic, simulator-grounded study of inference-time compute scaling for agentic UVM testbench synthesis targeting AMBA protocol bridges. Four inference strategies---zero-shot (\$0.04), raw-feedback iterative (\$0.81), structured inner-loop (\$0.95), and full dual-loop agentic synthesis (\$1.06)---are evaluated across four AMBA targets (Protocol Complexity Index 11.8--22.6) using the Protocol Verification Completeness (PVC) metric, functional coverage, and a six-class UVM error taxonomy (E1--E6). Our central finding is that structured diagnostic feedback acts as an inference-time compute multiplier: the structured inner loop yields roughly $15\times$ more PVC per inference dollar than raw EDA-tool feedback (marginal efficiency, S1$\to$S2 vs. S0$\to$S1). The mechanism is fault targeting, not token reduction: raw compiler feedback drives the testbench to compile but cannot surface functional protocol faults that leave the code syntactically valid, whereas structured E1--E6 bundles diagnose and resolve them. All outputs are judged by a cycle-accurate Mentor Questa toolchain rather than by model self-assessment; the dual-loop strategy reaches PVC $\geq$ 0.95 with zero scoreboard errors on every target, and the generated environments surfaced a latent RTL bug in two of the four reference designs. These results establish a principled compute-quality trade-off curve for LLM-aided verification environment generation and motivate inference-budget-aware deployment strategies for industrial EDA workflows.