Papers for

image dataset curators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vl-dedup improves deduplication of image and text data in clusters

VL-Dedup: Graded Multimodal Entity Resolution for Image-Text Data Deduplication

Abstract: Image-text corpora contain copies of one image, captions that describe one image in different ways, captions reused for different images, and captions that do not match their images. Each case calls for a different curation action. Deduplication and multimodal entity resolution (ER) only decide whether records are the same, and they are evaluated on labeled pairs. Yet pair scores do not tell which records end up in the same cluster or which useful captions get deleted. We therefore formulate image-text deduplication as graded multimodal ER, which decides visual identity, caption handling, and mismatches for each pair, and we evaluate it at three levels: pairs, clusters, and the retained set. Our method, VL-Dedup, groups records of the same visual entity into clusters and caps the size of each cluster. Within each cluster, a vision-language model (VLM) and CLIPScore decide which captions to keep and which to flag. We also build VL-DedupBench, with 50,000 COCO and SOP records and 66,788 labeled pairs. On it, a classifier that selects no false edge among the labeled pairs still joins millions of record pairs from different entities. Spending VLM calls on the pairs inside clusters, rather than on pairs with uncertain scores, gives better caption decisions from the same model. Even when almost all clusters are correct, keeping one record per cluster loses facts in many deletions, while the deletions of VL-Dedup lose much less information.

Mon 2 NovComputer Vision and Pattern Recognition
The gist
Large collections of image and caption pairs often contain duplicates or mismatched pairs, such as the same image with different captions or captions reused for different images. The authors created VL-Dedup, a method that groups images visually into clusters, controls cluster size, and uses specialized models to decide which captions to keep or flag. Unlike traditional methods that judge pairs independently, VL-Dedup looks at the entire cluster to make smarter decisions about duplicates and caption quality. They also built a new benchmark dataset to test their approach, showing it reduces information loss while cleaning data.
Open → 2611.03080v1