Data Selection Through Iterative Self-Filtering for Vision-Language Settings
2026-06-22 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors address the problem of noisy data when training large vision-language models. They introduce a new method called Self-Filtering, where the model is trained on a changing dataset that picks cleaner and more diverse examples over time. By repeating this process of training and selecting better data, the model improves without needing extra data or pre-trained models. This approach helps the model learn better from imperfect datasets.
neural networksvision-language modelsdata filteringCLIPself-trainingnoisy datadataset curationbootstrappingmachine learningmodel training
Authors
Andrei Liviu Nicolicioiu, Sarvjeet Singh Ghotra, Morgane M. Moss, Aaron Courville
Abstract
The availability of large amounts of clean data is paramount to training neural networks. However, at large scales, manual oversight is impractical, resulting in sizeable datasets that can be very noisy. Attempts to mitigate this obstacle to producing performant vision-language models have so far involved heuristics, curated reference datasets, and using pre-trained models. Here we propose a novel, bootstrapped method in which a CLIP model is trained on an evolving, self-selected dataset. This evolving dataset constitutes a balance of filtered, highly probable clean samples as well as diverse samples from the entire distribution. Our proposed Self-Filtering method iterates between training the model and selecting a subsequently improved data mixture. Training on vision-language datasets filtered by the proposed approach improves downstream performance without the need for additional data or pre-trained models.