Improved multidimensional signal modeling with dual neural networks and wavelet fusion
Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion
Computer Vision and Pattern Recognition
Summary
Many computer techniques struggle to capture both smooth and detailed parts of images or sounds, often missing fine details. This paper introduces a method that uses two types of neural networks working together: one focuses on the smooth parts and the other on the fine details. They combine their outputs using a mathematical tool called wavelets to capture all important features. The method works well on many types of data, including images, videos, and 3D shapes, making the results clearer and more accurate.
What this means in practice
- •For computer graphics developers: Generate higher-quality 3D shapes and volume data by capturing both smooth and detailed features in implicit neural representations.
- •For video processing engineers: Improve video reconstruction fidelity by combining neural network branches tuned for different frequency details and fusing via wavelets.
Authors
Wen Yan, Ligen Shi, Jun Qiu, Haimiao Zhang, Lina Wu, Chang Liu
Abstract
Implicit Neural Representations (INRs) have emerged as a compelling paradigm for modeling multidimensional signals by mapping continuous coordinates to signal values. However, Multi-Layer Perceptrons (MLP)-based INRs inherently suffer from spectral bias, which favors low-frequency components and suppresses the reconstruction of essential high-frequency details. While existing techniques, such as Fourier feature mappings, mitigate this issue, they often rely on sensitive manual tuning and are prone to spectral artifacts. In this paper, we propose a spatial-frequency-aware INR framework that combines an MLP branch with a Kolmogorov-Arnold network (KAN) branch for complementary frequency-oriented modeling. The MLP branch provides a low-frequency-oriented representation of smooth structures, whereas the KAN branch complements localized variations and fine details. To coordinate the two branches, we integrate the discrete wavelet transform (DWT) and inverse discrete wavelet transform (IDWT) into the output fusion stage. The outputs of the two branches are decomposed into wavelet coefficients, and the corresponding coefficients are additively fused before inverse wavelet reconstruction. A wavelet-domain band-separation regularization further penalizes high-frequency responses in the MLP branch and low-frequency responses in the KAN branch, thereby encouraging complementary frequency-oriented behavior. Experiments on 1D signals, 2D images, 3D volumes and signed distance functions, videos, and 4D light-fields demonstrate the applicability of the proposed representation across the evaluated signal modalities. Results demonstrate improved reconstruction fidelity across the evaluated signal modalities.