Cluster attention speeds up solving complex physical simulations
Cluster Attention Neural Operators for Solving Parametric Partial Differential Equations
Artificial IntelligenceMachine Learning
Summary
Simulating how fluids or solids behave can take a long time because each setting requires fresh calculations. The authors found a way to speed this up using a special kind of neural network that keeps detailed information without slowing down. Their method groups parts of the problem dynamically while still looking at everything else in full detail. This helps the model predict physical behavior more accurately and faster than previous methods.
What this means in practice
- •For engineering simulation teams: Accelerate design simulations by quickly predicting fluid and solid behavior under varying conditions without running full computations each time.
- •For computational physics developers: Integrate efficient attention-based models that maintain spatial detail for simulations involving complex geometries and long-term predictions.
Authors
Ming Zhong, Antonio Colanera, Gianluigi Rozza, Zhenya Yan
Abstract
Traditional simulations of parametric partial differential equations (PDEs) rely on repetitive computations for each parameter, which makes high-fidelity design impractical. Neural operators address this issue by learning solution operators, accelerating parameter-space mapping by orders of magnitude. Recent Transformer-based neural operators attempt to capture global dependencies, but often at the cost of quadratic attention complexity. Transolver resolves this problem by projecting physical states into a reduced slice space for attention computation. Although fast, this projection sacrifices fine spatial information. Moreover, by operating in this reduced space with shared weights across attention heads, it may constrain the model's flexibility, thereby limiting its capacity to capture complex phenomena. To address these issues, we propose the Cluster Attention Neural Operator (CANO), which reformulates attention via a novel cross-attention mechanism that dynamically clusters queries while preserving full-resolution keys and values. This avoids slice compression loss and removes weight-sharing limits. At the same time, the model remains fast without losing global interactions. Empirically, CANO achieves state-of-the-art performance across canonical PDE benchmarks, covering fluid and solid dynamics (e.g., Navier-Stokes, Airfoil, Plasticity), irregular unstructured geometries (e.g., Pipe Turbulence, Composites), and long-term temporal rollouts. Across solid deformation and turbulent flow benchmarks, CANO achieves lower errors than baselines and exhibits strong geometric adaptability and temporal consistency.