DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection
2026-08-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors focus on improving how visual-language models learn from large datasets by choosing the best examples to train on. They create a new score called Data Intrinsic Consistency (DIC) that checks if images, instructions, and responses in a sample all fit together well. Using this score, their method (DICS) smartly picks training data that balances quality and variety. Experiments show their approach works better than existing methods and even outperforms training on the full dataset using much less data. They also made a large collection of samples (DICS-6M) to test and prove their method on a big scale.
Vision-Language ModelsVisual Instruction TuningData SelectionData Intrinsic ConsistencyVisual Information ConsistencyResponse Information ConsistencyMulti-modal Instruction CorpusDataset EfficiencyModel Fine-tuning
Authors
Yuyang Hong, Jinhui Guo, Jiaqi Gu, Lubin Fan, Ruixiang Wang, Kun Ding, Yue Wu, Shiming Xiang, Jieping Ye
Abstract
Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data. We further curate DICS-6M, a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date; remarkably, DICS reaches 94.52\% of the official InternVL3-8B-Instruct performance using less than 25\% of its reported training data. Code can be seen at https://github.com/cqu-student/DICS