Distance-aware uncertainty restored for spline networks in high dimensions
Draining Fictitious Knots: Restoring Distance-Awareness Guarantees for High-Dimensional Spline Networks
Machine Learning
Summary
Some neural networks called Kolmogorov-Arnold Networks (KANs) use special functions called splines to predict data and know when they might be wrong. But in high-dimensional data, these networks sometimes think they are very sure in places where they actually have no real information, due to something called fictitious knots. The authors studied why this happens and created a method called drainage that fixes this problem by making uncertainty grow as you move away from real data points. This makes the networks better at knowing when they might be wrong, even in complex, high-dimensional cases.
What this means in practice
- •For machine learning engineers: Use drainage uncertainty to improve error estimates for spline-based models in high-dimensional tasks without heavy computational costs.
- •For computer vision teams: Deploy improved uncertainty estimates when modeling complex high-dimensional data like face images for safer prediction decisions.
Authors
Masoud Ataei, Mohammad Javad Khojasteh, Vikas Dhiman
Abstract
Kolmogorov-Arnold Networks (KANs) with spline activations have recently shown promise for interpretable function approximation. Distance-Aware Error for Kolmogorov Networks (DAREK) introduces a computationally efficient bottom-up approach to uncertainty quantification by equipping KANs with distance-aware error bounds; yet, in high-dimensional settings, the theoretical guarantees can be weakened by the emergence of fictitious knots. Inspired by the Kolmogorov-Arnold representation theorem, DAREK adopts a componentwise formulation in which each input dimension is treated separately; as a result, induced knot locations may appear in the combined input space without corresponding to actual training data. These fictitious knots mislead the DAREK uncertainty estimator into reporting low uncertainty far from any real observation, violating the distance-awareness guarantee. We identify this failure mode precisely, characterize its geometric structure, and propose a drainage uncertainty mechanism that restores distance-awareness by constructing a monotonically decreasing uncertainty path from any fictitious knot region toward the nearest real knot. The proposed drainage method provides a practical heuristic correction that mitigates the fictitious-knot failure mode while restoring theoretical distance-awareness in high-dimensional settings. Experiments on a 2D synthetic benchmark and a 100-dimensional face dataset show that drainage raises sampled distance-awareness (SDA) from 85% to 98-99%, matching Gaussian processes at lower computational cost.