Spatial hearing improves through feedback using the free energy principle
Framing Spatial Hearing within the Free Energy Principle: A Hierarchical Model of Perceptual Learning
SoundArtificial Intelligence
Summary
People use hearing to figure out where sounds come from, but this is difficult because sounds and environments change all the time. The authors show a way to understand how listeners learn to localize sounds better by updating their internal beliefs after getting visual feedback. Their model explains how different parts of hearing sensitivity improve over time, especially for vertical sound direction. This research helps connect theories about brain function to how we learn spatial hearing in everyday situations.
What this means in practice
- •For audio technology engineers: Design hearing aids that adaptively improve users’ spatial hearing by incorporating visual feedback to fine-tune sound localisation.
- •For virtual reality developers: Create immersive audio environments where users’ spatial hearing adjusts dynamically through feedback loops to enhance realism.
Authors
Roberto Barumerli, Nils Meyer-Kahlen, Lorenzo Picinali
Abstract
Spatial hearing relies on dynamic internal representations to decode spatial information from ever-changing acoustic conditions. A fundamental example is sound localisation, where listeners infer the direction of an invisible sound. Bayesian inference provides a computational account of such a localisation process; however, modelling efforts are largely limited to static listening scenarios. Instead, the free energy principle (FEP) describes perception, learning, and action as joint minimisation of variational free energy (VFE), providing a principled framework to account for dynamic conditions. Here, we take a first step toward applying the FEP to spatial hearing. We characterise how listeners' beliefs evolve across trials, in which they estimated sound direction and received subsequent visual feedback on each trial. Listeners are described by a hierarchical generative model where visual feedback drives gradient descent on VFE, updating beliefs over three independent quantities: lateral and polar localisation precisions and hemifield disambiguation. The model captures individual learning trajectories and reveals components with different dynamics: lateral precision improves, as does hemifield disambiguation, consistent with increased sensitivity of the auditory pathway in decoding vertical direction. Notably, polar precision also increases, most for listeners with the lowest initial precision, suggesting that the observed vertical improvement is driven by both hemifield disambiguation and polar precision. This work establishes spatial hearing as a tractable domain for FEP and motivates future closed-loop experiments with full action-perception cycles.