Agentic co-design speeds AI chip and software optimization jointly
Learning to Formulate: Agentic Hardware-Software Co-Design of AI Infrastructure through Progressive Optimization and Verification
Hardware ArchitectureArtificial IntelligenceDistributed, Parallel, and Cluster Computing
Summary
Designing the hardware and software for AI systems usually happens in separate layers, which limits how much they can improve together. The authors create a process called Formulaic where an AI agent writes the design problem, a solver optimizes it, and then verification steps check correctness. This method lets the agent refine the problem step-by-step, improving both chip and software together. Their tests show faster performance on real and simulated GPUs compared to existing approaches.
What this means in practice
- •For ai hardware designers: Improve AI chip performance by jointly optimizing chip design and software coordination using the Formulaic agentic formulation approach.
- •For gpu software engineers: Achieve faster AI model execution by applying agent-written optimization templates that better align hardware and software capabilities.
Authors
Yi Ding, Li Shang
Abstract
AI infrastructure, the chips and software that train and serve AI models, is designed in layers. Layers make the system manageable, but they also fix which choices an optimizer can consider. Gains that need the chip and its software to change together are excluded before optimization begins: the formulation leaves them out. Language-model agents make it possible to revisit these boundaries, but an agent's proposal cannot show by itself that a design is correct or optimal. We present Formulaic, a flow that divides the work. The agent writes the formulation, a mathematical solver optimizes within it, and verification and measurement check its assumptions and results. The agent refines the problem step by step, from the computation graph toward the chip. Each step is a template: a family of programs with the conditions under which they are correct. Wall relaxation searches the chip by pricing the hardware limits that bind the software. This paper takes a first step, through separate studies rather than one end-to-end run. On NVIDIA GB10, templates the agent wrote run complete Qwen3-8B layers $1.075\times$ faster than SGLang. In a simulated Ventus GPU study, joint design cuts the worst-scenario cycles of Qwen2.5-0.5B to 0.428 of the reference, against 0.934 for software alone. On a five-step GEMM refinement chain, reusing proofs makes checking $2.51\times$ faster.