Framework guides microscope decisions to speed experiments and improve learning
Building heterogeneous multistep workflows in automated microscopy: reward design and knowledge transfer for autonomous structure property discovery
Machine Learning
Summary
Microscopes can collect lots of data, but deciding exactly what to measure and when can be tricky. The authors present a system that helps microscopes make smarter choices during experiments by considering costs, previous knowledge, and instrument changes over time. Their system was tested using real spectroscopy data, showing it can choose different measurement types and know when to recalibrate the microscope to keep results accurate. This approach also helps understand when simple measurement rules work well and where more complex decisions add value.
What this means in practice
- •For microscope instrumentation teams: Design microscope software that adapts scanning and data collection strategies to experimental conditions and instrument status automatically.
- •For laboratory automation engineers: Improve autonomous experiment planning by incorporating explicit cost and knowledge transfer models for better resource use and data quality.
Authors
Sergei V. Kalinin
Abstract
Autonomous microscopy can turn fast acquisition into scientific progress when it decides not only where to measure, but also what information to acquire, at what scan rate, and when to recalibrate the instrument. We develop a budget constrained framework for such multistep structure-property workflows. It combines explicit scientific rewards, action costs, instrument observation models, transfer of pre experiment knowledge into the online campaign, and evaluation kept independent of the controller. The transferred knowledge includes predictor accuracy and computational cost, empirical learner maturity functions that weight model-derived rewards, and drift characteristics established before controller training. Using a real electron energy-loss spectroscopy dataset, we run two complementary campaigns. In the first, the controller chooses between imaging and spectroscopy, image size, and scan speed. In the second, the instrument drifts and the controller also decides when to recalibrate. The campaigns show that response discovery and structure-property learning favor different surrogate models and acquisition strategies. They also show that simple acquisition rules form strong, practical baselines. Independent error analysis reveals where model uncertainty is and is not a reliable learning signal. In the drift campaign, explicit registration greatly improves positional accuracy, and the framework measures what this costs in spectroscopy throughput. Together, these results provide a reproducible way to design, compare, and audit autonomous experiments, and they define what microscopes must provide for deployment: independently executable actions, characterized scan and detector responses, timestamped registration telemetry, and complete action records.