Automated verification catches bugs in ai-written hardware designs
Will AI mess my RTL? A snapshot-based approach to verifying AI-written RTL
Software EngineeringHardware Architecture
Summary
As people use artificial intelligence to write parts of computer chips, it becomes important to make sure these changes don’t break anything. The authors created a fully automated tool called SVApshot that checks and fixes the rules describing how chip parts should behave. This tool uses AI to write tests and updates them when something goes wrong, without needing human help. Their tests show it finds bugs added by both humans and AI, helping make AI-designed hardware more reliable.
What this means in practice
- •For hardware design teams: Validate and repair AI-generated hardware code by automatically creating and checking formal assertions to ensure designs maintain correct functionality.
- •For hardware verification engineers: Generate comprehensive assertion-based test suites that detect subtle bugs in both manual and AI-modified RTL designs, improving bug coverage.
Authors
Bernat Homs, Oscar Palomar, Miquel Moreto, Marcelo Orenes-Vera
Abstract
With the growing interest in using AI (Artificial Intelligence) for RTL (Register-Transfer Level) hardware development, robust and comprehensive verification has become more important than ever. As Large Language Models (LLMs) increasingly assist in creating and modifying RTL designs, ensuring that these changes preserve existing functionality is paramount for building trust in AI-driven workflows. We present SVApshot, a fully automated framework that leverages LLMs to generate and systematically attempt to correct SystemVerilog Assertions (SVA) for RTL modules, completely removing the human from the generation loop. SVApshot introduces a novel snapshot methodology that captures the current functionality of a design as a comprehensive set of formal assertions, creating a regression suite for validating future changes--whether made by humans or AI. The framework features automated duplicate detection, assertion set expansion, and iterative repair of failing assertions using LLM-guided debugging with formal verification feedback. Our experimental evaluation across diverse RTL modules demonstrates high coverage for complex modules, with successful detection of both manually-injected and AI-introduced bugs. Additionally, this framework opens the door for automatic generation of formal testbenches to benchmark LLMs for hardware design, moving beyond binary pass/fail metrics towards nuanced assertion-based scoring.