Learnable spectral activations improve neural signal fitting efficiency
Learnable Spectral Activations
Machine LearningComputer Vision and Pattern Recognition
Summary
Neural networks often work by combining different frequency patterns to recreate signals like sounds or images. This paper presents a new way to adjust how these patterns are combined by letting the network learn how to shape them directly, instead of using fixed designs. The authors show that letting the network learn these frequency combinations makes it better at representing varied and complex signals. This improvement helps the network learn faster and produce higher-quality results across different tasks like audio reconstruction and 3D scene rendering.
What this means in practice
- •For audio engineers: Enhance audio signal reconstruction quality and efficiency by using networks that adaptively shape frequency components during training.
- •For computer vision developers: Improve image and 3D scene reconstructions by enabling neural networks to learn optimal spectral activations tailored to spatial details.
Authors
Tamir Shor, Or Litany, Alex Bronstein
Abstract
Implicit neural representations (INRs) are shaped by the spectral structure induced by their input encodings and activation functions. Existing methods improve fitting primarily by modifying which frequencies are available to the network, through coordinate encodings or periodic nonlinearities. However, frequency access is not the only bottleneck: signals with localized or spatially varying structure require the network to efficiently compose frequencies into multi-harmonic internal responses. We introduce learnable spectral activations (LSA), which replace fixed neuron-level nonlinearities with a residual truncated Fourier series whose harmonic amplitudes are learned during training. LSA does not expand the asymptotic function class. Instead, it changes the factorization of the representation: linear weights select features while activation coefficients control spectral shaping, and the two are updated by separate gradients. Because the activation output is affine in the coefficients given fixed pre-activations, spectral tuning becomes a more direct subproblem compared to architectures where it is entangled with feature selection. Empirically, this factorization concentrates more target-signal energy in the leading eigenmodes of the neural tangent kernel, consistent with improved optimization behavior. Across audio, image, neural radiance field, and neural acoustic field tasks, LSA also improves reconstruction quality.