Graph neural network improves wide-area 6DoF spatial audio rendering
Graph Neural Network for Wide-Area 6DoF Ambisonic Rendering
Sound
Summary
Capturing and recreating 3D sound from a few microphones spread out in a space is tricky because mixing audio signals from faraway spots can cause distortions. The authors show how using a graph-based neural network to smartly combine measurements based on their positions and acoustic properties leads to better sound quality at new listener locations. Their method keeps the original sound's energy and direction more accurately than older ways that just use fixed geometric rules. They tested their approach in rooms and setups the network had never seen before, and it worked better.
What this means in practice
- •For spatial audio engineers: Produce more accurate 3D audio experiences by improving how sparse Ambisonic recordings are combined for listener movement.
- •For virtual reality developers: Enhance immersive soundscapes in VR environments by refining audio rendering techniques that adapt to user positions using learned interpolation weights.
Authors
Gioele Greco, Filippo Marri, Mirco Pezzoli, Fabio Antonacci
Abstract
Wide-area 6DoF rendering from sparse Ambisonic recordings is commonly addressed by interpolating parametric spatial descriptors rather than the waveforms themselves. This preserves the fidelity of a nearby measured signal and avoids phase artifacts caused by mixing distant arrays. However, existing DirAC-based approaches rely on geometric interpolation weights that do not depend on the acoustic scene observed by the microphones. We propose a lightweight Graph Neural Network (GNN) that refines only this interpolation stage. The known Ambisonic arrays and the query position are represented as a graph whose nodes contain local DirAC features and whose edges encode relative distance and directional relations. The network predicts positive query-conditioned weights over the measured arrays, while the target energy, diffuseness, and direction of arrival are reconstructed by an analytical DirAC layer. Experiments on unseen rooms, unseen speech signals, and unseen array configurations show that the proposed weighting improves energy reconstruction and radiation-pattern similarity with respect to baseline interpolation.