LLM agents produce physical models and enable better model acceptance
LLM Agents as Suppliers of Physical System Models: Calibration and Acceptance in Modelica
Machine LearningArtificial Intelligence
Summary
Getting a good model of a device can be tricky because descriptions and data don't always fully specify what the model should look like. The authors had AI agents build models from parts based on descriptions and data, then checked how well these models matched expectations. They found that many models fit the data but still had errors or missing physical principles, so acceptance should consider a range of models and additional checks like physical laws. When the criteria for acceptance were clearer, the AI agents produced better models. This approach helps understand when a model is truly good enough.
What this means in practice
- •For control system engineers: Automatically generate and refine models of physical devices from descriptions and data, improving acceptance in calibration tasks.
- •For industrial automation teams: Use LLM agents to build interpretable device models that include physical invariants, aiding fault detection in manufacturing systems.
Authors
Jörg Frochte, Herbert Schmidt
Abstract
To procure a model of a device, a client hands a supplier a description, sometimes with recorded data or a target response, and accepts the model against this handover. We let LLM coding agents act as the supplier on handovers with data, building Modelica models from library components for four simulated systems and a real switching magnet, and ask what acceptance certifies. Several agents reach the noise floor, but a handover admits a set of models: the description leaves parts of the structure open, and the data narrow it only to the agreed accuracy. We therefore accept by membership in this set, not by matching one reference, and add layers above behaviour: a physical invariant exposes an energy-creating model at the noise floor. Interpretability becomes a property of the delivered model, tested on a variant, read where none exists and shaped through the handover. Most failures are fits delivered with a systematic residual on their own fitting data, where the handover left open how close is close enough. In a follow-up, stating a threshold makes the agents compare against it, and more deliveries pass. The argument applies wherever a handover underspecifies a model meant to be read, not only run.