Ai agents with architecture knowledge improve hardware design efficiency
Do AI Agents Understand Computer Architecture?
Artificial IntelligenceHardware Architecture
Summary
It is unclear whether AI agents that improve hardware designs truly understand the underlying computer architecture or merely tweak parameters blindly. The authors created a test giving the same AI agent two versions of the hardware design problem—one with meaningful architectural information and one without—while keeping everything else the same. They found that agents using architectural knowledge performed better and used less simulation time, showing that meaning helps. However, some other techniques can reduce this advantage, suggesting that understanding architecture and structured feedback can substitute for each other.
What this means in practice
- •For hardware design teams: Use AI agents informed by architectural knowledge to design accelerators more efficiently with fewer costly simulation runs.
- •For machine learning engineers: Incorporate structured critique loops into AI design agents to improve optimization when architectural knowledge is unavailable.
Tested on simulated data.
Authors
Ambika Sharan, Grigory Chirkov, Soheil Abbasloo
Abstract
Agents are increasingly asked to design hardware, and increasingly reported to succeed. Such reports establish that a design improved; they cannot establish why. An agent that improves an accelerator may be reasoning about the machine, or may be searching competently over knobs whose meaning it never recovers -- and only the first transfers to the next architecture. Existing evaluations cannot tell the two apart, because they vary the agent while holding the framing of the problem fixed. We do the opposite. AutoTuring hands the same agent the same 15-dimensional accelerator space twice: once as named architectural knobs with simulator counters, once as anonymous variables on [0,1], with the evaluator, the legal space and the reachable optima held identical, so that the only thing that varies is whether the problem means anything. The gap between the two is the measurement. On a nine-kernel FP16 GEMM basket, meaning pays: the architect beats a modeled H200 by 5.4% and its blind counterpart by 12.3% on average, with 70.1% fewer simulator calls. It does not pay uniquely: a critic loop recovers most of that gap for the blind agent and buys the architect nothing, so architectural knowledge and structured critique behave as substitutes rather than as complements. We report these as preliminary findings -- five to six runs per condition on a single modeled accelerator -- and take the comparison itself, not the accelerator, to be the contribution.