Neural networks learn physical data better using susceptibility insights
Susceptibilities for Neural Networks Learning from Physical Data
Machine Learning
Summary
Understanding how neural networks learn from physical data is tricky because the data follows physical laws. The authors show that by measuring how sensitive a neural network is to changes in the data's underlying parameters—called susceptibilities—they can find out when certain kinds of data help the network learn best. They also suggest a way to create such parameters when none exist naturally, making it possible to track when networks develop specific skills during training. Their experiments back up these ideas, helping improve how data is chosen for training.
What this means in practice
- •For machine learning engineers: Identify which input features best help neural networks learn from physical data by measuring susceptibility to guide effective data selection.
- •For robotics developers: Monitor neural network training on sensor data to detect when specific capabilities like collision detection emerge using introduced physical parameters and susceptibilities.
Authors
Rohan Hitchcock, Gary W. Delaney, Jonathan H. Manton, Richard Scalzo, Jingge Zhu
Abstract
Neural networks that are trained on data arising from a physical system must somehow learn regularities induced by the underlying physical laws. In this setting, concepts from statistical physics provide powerful tools for analysing how physical laws influence the learning process. In this paper we explain how susceptibilities --- which measure the response of a neural network to perturbations of a parameter of the data distribution --- can be applied when learning from physical data. We show that such susceptibilities can identify the conditions under which specific input features are most informative for learning, guiding data selection, a prediction we validate experimentally. For distributions lacking a natural parameter, we propose introducing one via a physically meaningful scalar function of the physical state, and demonstrate that susceptibilities track the emergence of specific capabilities (such as collision detection) during training.