WaferTetris boosts chip use and speeds AI tasks on wafer-scale hardware
WaferTetris: Efficient Freeform Mapping on Wafer-Scale Accelerators
Hardware ArchitectureComputation and Language
Summary
Wafer-scale computer chips have thousands of tiny processors that work together, but putting tasks onto these chips efficiently is tough. Most ways to organize these tasks force fixed shapes and layouts, which wastes power and memory. The authors created WaferTetris, a method that fits tasks like Tetris pieces onto the chip more flexibly. This better use of space helps run big AI models and scientific calculations faster and with more memory. Their tests show WaferTetris can increase speed and support longer AI model contexts than other methods.
What this means in practice
- •For chip designers: Map complex AI model tasks across wafer-scale chips with higher speed and memory use by flexibly allocating processing elements.
- •For high-performance computing teams: Accelerate scientific simulations on wafer-scale hardware by optimizing the placement of computational tasks beyond fixed templates.
Authors
Dayou Du, Congjie He, Andrew Boutros, Luo Mai, Vaughn Betz, Jianyi Cheng
Abstract
Wafer-scale accelerators offer massive compute capacity through hundreds of thousands of processing elements (PEs) interconnected across a single wafer, but realizing this potential requires efficient application mapping. Their scale makes programming these systems and mapping applications onto them challenging. Existing approaches rely on template-based mappings to maintain productivity, but these templates restrict the mapping design space. In particular, they impose fixed geometric layouts over the PE grid, often leaving compute and memory resources underutilized. In this work, we present WaferTetris, a system that removes these geometric restrictions through freeform mapping. Its Tetris-like strategy flexibly packs computation across the wafer, improving PE utilization and usable memory capacity while preserving PE-task code. We develop a recurrence-based throughput model that captures branch-and-join dependencies to guide spatial parallelism, and an architecture model with per-color gates that enable existing FPGA place-and-route algorithms to enforce WSE routing constraints. We evaluate WaferTetris on Cerebras WSE-3 using LLM decoding and scientific stencil benchmarks. For LLM decoding, it achieves up to $4.42\times$ the system throughput of a template-based automatic mapping baseline on the same fabric size. Compared with the manually optimized WaferLLM, WaferTetris supports $1.8\times$ the context length for LLaMA3.1-8B (80k versus 44k), while delivering equal or higher per-user throughput at matched context lengths, up to $1.10\times$.