Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors propose TriNoL, a method to improve how vision foundation models (VFMs) adapt using partly labeled data. Instead of treating all unlabeled data the same, their system sorts it into three groups based on confidence: reliable, uncertain, and likely wrong. Each group trains a specialized small module (LoRA expert) separately, while keeping the main model fixed. This helps the model handle noisy and unclear labels better without adding much extra training cost.
vision foundation modelssemi-supervised learningpseudo-labelsLoRAmodel adaptationnoisy labelsconfidence estimationlow-rank adaptationexpert models
Authors
Xuanyu Liu, Zheng Fang, Hongyang He, Yundi Hong, Daizong Liu
Abstract
Semi-supervised adaptation of vision foundation models (VFMs) commonly freezes the pretrained backbone and updates lightweight modules such as LoRA. However, pseudo-labels have mixed reliability, and a single LoRA adapter must absorb reliable, ambiguous, and noisy gradients in the same low-rank space. This can make VFM adaptation sensitive to pseudo-label noise. We propose \textbf{TriNoL}, a \textbf{Tri}ple-expert learning framework from \textbf{No}isy \textbf{L}abels for semi-supervised VFM adaptation. TriNoL routes unlabeled samples into three confidence regions and assigns them to three LoRA experts: a Positive Expert for high-confidence pseudo-labels, an Alignment Expert for medium-confidence ambiguous samples, and a Negative Expert for low-confidence noisy samples. The VFM backbone remains frozen, and only the LoRA experts and classifier head are updated. By separating different pseudo-label reliability regions into specialized adaptation paths, TriNoL improves robustness to noisy supervision while keeping the training cost low.