Apexa ensures correct automation of synchrotron data processing
APEXA: Execution-Integrity Enforcement for Multi-Agent LLM Automation of Synchrotron Data Reduction
Artificial Intelligence
Summary
Processing huge amounts of data from synchrotrons, which are special particle accelerators, is complicated and requires expert knowledge. Some AI systems called large language model agents can help automate this, but they sometimes invent steps they never did, causing errors. The authors created APEXA, a system that runs multiple automated tools while making sure every result comes directly from a real action, not a guess. They tested it on real data and built a way to check if it works correctly on many different tasks.
What this means in practice
- •For light source facility operators: Automate detector calibration and data integration reliably with executable results enforced to avoid incorrect or fabricated steps.
- •For industrial sensor data teams: Use execution-integrity enforcement to ensure automated processing results are always based on actual executed steps rather than generated text.
Authors
Pawan K. Tripathi, Hemant Sharma, Andrew Chuang, Mathew J. Cherukara
Abstract
Synchrotron data reduction, detector calibration followed by azimuthal integration of terabyte-scale diffraction series, is a multi-step, expert-bound bottleneck that increasingly limits the science rate of user facilities. LLM agents promise to collapse it, but driving a real pipeline with a stochastic model creates a failure mode chat benchmarks cannot see: an agent can report a calibration that was never computed. Correctness here is a property of what executed, not of the transcript. We present APEXA, a deployed multi-agent framework (61 tools over heterogeneous compute, run as a single reasoning loop) automating calibration and integration from natural language at a major light source. We make three contributions. First, execution-integrity enforcement: a deterministic tool-layer guard that refuses to surface any result not backed by an executed tool call, with a parser tolerant of cross-model tool-call format drift: in deployment, a frontier model fabricated a complete calibration-comparison report for commands that never ran, which the guard converts to an explicit non-result; the same code gates an optional motor-control surface at 0/200 adversarial violations against a simulated IOC, versus 15/200 for an equivalent safety prompt. Second, we release APEXA-Bench, an evaluation harness of 58 facility tasks (50 base plus an 8-task cross-detector slice) organized by a four-class physical-consequence taxonomy, the first benchmark axis we know of separating a wasted compute cycle from a damaged instrument; its cross-detector grading against NIST-traceable lattice constants surfaced two latent pipeline bugs. Large-scale agent scoring is left to a full-length study. Third, we validate APEXA on real beamline data: from one natural-language prompt it recovers detector geometry and integrates a full attenuation/exposure sweep. We release the framework, harness and traces.