FlatManifold: Robust Continual Learning under Severe Label Noise and Domain Shifts via Intrinsic Manifold Flattening
2026-07-06 • Machine Learning
Machine Learning
AI summaryⓘ
The authors address the problem of learning from data streams where both the environment changes and labels can be very noisy. They propose a new method called FlatManifold that transforms data into a special flattened space using mathematical tools like kernels and projections to reduce the impact of incorrect labels. This approach also helps the model remember past knowledge better by using information from previous data sessions. Tests on robotics datasets show that FlatManifold works well even when almost half the labels are wrong and the conditions change a lot over time, outperforming standard methods.
continual learninglabel noisekernel trickNyström methodReproducing Kernel Hilbert Space (RKHS)manifold flatteningcatastrophic forgettingridge regularizationcovariance matrixdomain shift
Authors
Rai Hisada, Kanji Tanaka
Abstract
In non-stationary streaming environments, simultaneously adapting to complex, non-linear domain shifts via continual learning while mitigating the catastrophic effects of severe, uncalibrated label noise poses a fundamental mathematical challenge. In this paper, we propose \FlatManifold{}, a novel, streamlined robust continual learning framework that utilizes a Nyström manifold flattening map based on the kernel trick and projection onto an orthogonalized Reproducing Kernel Hilbert Space (RKHS). Unlike traditional methods that rely on complex, error-prone sample-filtering pipelines, the proposed approach exploits the intrinsic mathematical robustness of the flattened space itself. By mapping feature distributions onto a fixed orthogonal target topology with a ridge regularizer, the framework naturally smoothes and counteracts the influence of extreme label noise during the optimization process. Concurrently, catastrophic forgetting is prevented via a continual topology brake term that leverages the covariance matrix of past experiences. Extensive evaluation on real-world multi-session robotics datasets demonstrates that even under severe conditions featuring 40\% symmetric label noise, \FlatManifold{} successfully mitigates gradient corruption. Under extreme cross-session domain shifts spanning various seasons and lighting conditions, the proposed framework establishes high generalization capabilities, significantly outperforming standard sequential optimization baselines and proving that structural linearization itself serves as a powerful mathematical barrier against distributed label corruption.