Text prompts do not clearly guide neural models for physical simulations
Does Text Steer Neural PDE Surrogates? A Controlled Diagnostic with OperatorCLIP
Machine Learning
Summary
Lower error rates when using text-based instructions in neural models do not prove the models understand the text's meaning. The authors tested this by comparing different ways of including text prompts in models simulating physical systems like fluid flow. They found that just adding text did not reliably help the models or show that they used the text meaningfully. This means more careful tests are needed before assuming such models truly understand or use text descriptions.
What this means in practice
- •For neural model developers: Evaluate text-conditioning methods on physical simulation models to verify if text meaning is really used or just incidental.
- •For simulation software engineers: Use controlled tests to decide whether to integrate text prompts into PDE surrogates for industrial or research simulations.
Authors
Aadi Dash, Lennon J. Shikhman, Michael Galarnyk
Abstract
Lower error from a text-conditioned neural surrogate does not, by itself, show that the model uses the meaning of the text. We examine this attribution problem with OperatorCLIP, comparing an unconditioned FNO, a constant-sentence FiLM control, and a fixed task description trained with contrastive alignment. Three-seed experiments cover Darcy2D, ShallowWater2D, and three-dimensional compressible Navier-Stokes (CNS3D). Constant conditioning has lower mean test error on both 2D tasks. Relative to this control, task text plus alignment has a similar mean on ShallowWater2D and CNS3D and a higher mean on Darcy2D; these descriptive comparisons have substantial seed uncertainty. The latter comparison changes both prompt content and loss, so it isolates neither effect. The text encoder is trained from scratch, and each conditioned model sees only one description during training. In this regime, pairwise InfoNCE cannot identify matched pairs and has minimum $\log B$. Prompt interventions show no reliable semantic ordering. This methodological caution demonstrates why pathway controls are needed; it neither establishes semantic competence of the encoder nor tests the effectiveness of text under varying physical context.