LLM adapts neural operator tuning using both prior knowledge and feedback
Prior or Feedback? What an LLM Uses When Adapting Neural Operators
Artificial IntelligenceMachine Learning
Summary
Large language models (LLMs) can help tune neural networks used to solve math problems called partial differential equations. This paper shows that these models don't just pick settings based on what they know in advance, but also adjust their choices after seeing how well earlier attempts worked. The researchers tested how the LLM makes decisions before and after getting feedback and found it uses both its initial biases and the feedback to improve results. So, the LLM combines what it expects with what it learns from experiments to pick better tuning settings.
What this means in practice
- •For machine learning engineers: Choose neural network fine-tuning settings more effectively for solving varied scientific equations using LLM-guided adaptation that learns from initial runs.
- •For computational physics teams: Improve accuracy in numerical simulations of physical systems by leveraging LLMs that adapt neural solvers based on early feedback.
Authors
Julian Chan, Javier Mora Jimenez
Abstract
Do LLM scientific agents rely only on their initial task context, or do they adapt their decisions in response to experimental feedback? We study this question in neural operator adaptation, where a large language model (LLM) selects fine-tuning configurations under a limited trial budget. Across transfers within and between partial differential equation (PDE) families, the LLM achieves lower held-out test nRMSE than random search and Bayesian optimisation in nearly every matched comparison. Endpoint performance alone cannot distinguish what happens, so we verify each attribution with controlled interventions. Before observing any validation score, the LLM's first configuration already ranks near the top of the corresponding random-search pool, indicating a useful initial bias. A complementary cold-start intervention shows that the selected base learning rate shifts with the PDE description. Once feedback becomes available, reassigning validation scores among evaluated configurations changes the next proposal in every case tested, whereas a value-preserving rewrite produces no comparable aggregate effect. These interventions establish that the LLM's decision-level actions respond to the given task and observed outcomes, showing that it combines a task-dependent prior with sensitivity to experimental feedback.