Transformer improvements use wall distance to boost 3D flow predictions
Distance-Aware Attention and Wall-Distance Expert Routing for Transformer-Based 3D Flow Prediction
Artificial Intelligence
Summary
Predicting fluid flow in 3D meshes is hard because each point interacts differently depending on its location, especially near walls. The authors improved transformer models by making them aware of how far each point is from a wall, changing how they gather and process information. Their new methods help the models focus on important details near boundaries and recover details farther away. This results in more accurate predictions of pressure and velocity in complex flows, even for new shapes the models haven’t seen before.
transformer3D flow predictioncross-attentionmeshboundary layerwall distancefeed-forward layermixture-of-expertspressure errorvelocity error
Authors
Sanghyeon Kim, Sunwoong Yang, Namwoo Kang
Abstract
Transformer surrogates for 3D flow prediction compress an industrial mesh into a small set of tokens from which every prediction point reads. Two operations follow: the retrieval step in which a point gathers information from the compressed representation, and the feed-forward layer that transforms what it retrieved. In current backbones both are blind to where the point sits in the flow. We condition both on wall-related physical signals. Distance-aware cross-attention (DA-CA) reshapes each volume query by its wall distance before retrieval, so that a point deep in the boundary layer draws different geometric information than one in the outer flow. Surface-volume mixture-of-experts (SVMoE) replaces the shared feed-forward layer with a small set of experts, routed by wall distance for volume points and by local geometry for surface points. Neither mechanism is tied to one architecture, so we apply both unchanged to AB-UPT and Transolver-3. On DrivAerML with 50 training cases, DA-CA reduces the volume pressure error by 10.1%, and DA-CA and SVMoE together reduce it by 12.5%; DA-CA improves the near-wall region at some cost in the far region, which SVMoE recovers, and the volume experts settle into near-wall, transition, and free-stream bands without routing supervision. Retrained on 300 cases, the conditioning improves every field quantity, reducing volume pressure and velocity errors by 33.1% and 18.6% on AB-UPT and by 21.4% and 21.3% on Transolver-3. Under Leave-One-Body-Out evaluation on DrivAerNet++, it reduces the volume pressure error on unseen body types by up to 14.2%.