Papers for

digital asset managers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language model embeddings improve image clustering accuracy

Image Clustering with Vision-Language Model Embeddings

Abstract: Pretrained vision-language models provide image representations whose suitability for clustering may vary with feature extraction, image domain, and partitioning algorithm. We compare nine representation pipelines across seven image benchmarks and four clustering configurations, yielding 252 baseline combinations. Eight pipelines extract visual features, including two vision-only references; the ninth generates descriptions and embeds the resulting text. With agglomerative clustering (Ward linkage) fixed, Qwen3-VL achieves the highest adjusted Rand index (ARI) on five datasets, including 0.925 on MNIST, while BLIP-2 leads on CIFAR-10 and CIFAR-100 with ARIs of 0.925 and 0.591, respectively. A separate sensitivity analysis compares raw and L2-normalized embeddings at their original dimensionality across all 63 dataset-representation pairs. Normalization preserves the highest-ARI representation in every dataset, increases ARI in 35 pairs, decreases it in 26, and leaves two numerically unchanged. On the SVHN dataset, the highest ARI remains low, rising from 0.097 to 0.098 after normalization. The evaluated caption-mediated pipeline obtains relatively high ARI on isolated digits but does not consistently exceed direct visual features. These findings support selecting and evaluating complete representation-clustering pipelines for the intended domain, with explicit attention to extraction and normalization choices.

Fri 23 OctComputer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
The gist
Clustering images—grouping similar pictures together—is tricky and depends on how images are represented and processed. The authors tested nine ways of representing images with vision-language models across many datasets and clustering setups. They found some methods that work better for certain types of images, especially when features are normalized. Their experiments also showed that describing images with text and then clustering based on those descriptions isn't always better than using visual features directly. This study helps guide how to pick and prepare image features for clustering tasks.
Open → 2610.33265v1

Scalable deep feature encoding boosts scene image retrieval accuracy

Rethinking Fisher Vectors for Second-order Multi-scale Deep Feature Aggregation in Scalable Scene Retrieval

Abstract: Aggregating CNN responses into a compact discriminative vector, given their lack of geometric invariance, is a challenge. To address this challenge, we propose a codebook-free method to encode second-order information of multi-scale CNN features within the Fisher Kernel framework. We use a scalable retrieval pipeline to show the efficacy of the proposed vector. The increased discriminative power is confirmed by experimental evaluation of our method on two difficult benchmarks (INRIA Holidays, UKBench). The significant improvement in retrieval accuracy over state-of-the-art approaches comes with only a slight dimension increase.

Thu 22 OctArtificial Intelligence
The gist
Finding similar photos in large collections is hard because computers struggle with changes in viewpoint and scale. The authors introduce a new way to combine information from multiple scales of deep neural network features, capturing richer details without relying on predefined codebooks. They encode this information using a method related to Fisher Vectors, improving how images are compared and retrieved. Tests on challenging image datasets show their approach outperforms previous methods with only a small increase in data size.
Open → 2610.31146v1

AI tool detects copyright leaks in generated videos by matching images

Copyright Leakage Detection in AI Generated Videos

Abstract: Video generation models can create realistic-looking video footage from images and text prompts. Some of the recent advances have raised concerns about the reproduction of copyrighted visual content. However, copyright leakage detection in generated videos remains largely unexplored. In this paper, we introduce a new benchmark data set for copyright leakage detection, comprising 50 copyrighted concepts, each with 5 images. For each concept, we have 5 videos created using the image+prompt video generation model and the prompt-only video generation model. Following that, we develop an AI toolset based on a contrastive retrieval framework that formulates copyright leakage detection as a video-to-image retrieval problem by matching aggregated video frame embeddings against a database of copyrighted image embeddings. Our method uses a single similarity threshold and retrieves the most similar copyrighted reference as supporting evidence. We conducted experiments showing strong retrieval and detection performance, achieving 99.6\% detection accuracy with only a single false negative. These results show the effectiveness of the retrieval-based method for scalable and explainable copyright leakage detection in AI-generated videos. Data can be accessed publicly from https://zenodo.org/records/21386117 and the source code for this project is publicly available at https://github.com/Shaif95/video-cp-Leakage.

Tue 20 OctComputer Vision and Pattern Recognition
The gist
AI video generators can create realistic videos from pictures and text, but sometimes they accidentally copy copyrighted images. This paper presents a new set of videos and images to test if AI videos copied copyrighted content. The authors created a method to compare videos to a database of copyrighted images by looking at video frames and finding the closest matches. Their method works very well, catching nearly all copies while showing evidence for each match.
Open → 2610.28524v1

Attributes improve person re-identification by adapting query space

In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification

Abstract: Person re-identification (ReID) ranks an image gallery using either an image query or a textual query. In image-to-image (I2I) ReID, viewpoint, illumination, occlusion, and background changes can make distractors appear more similar than the true match. In text-to-image (T2I) ReID, incomplete or ambiguous descriptions may omit key visual evidence. We study this problem for vision-language model (VLM) based ReID, where attributes such as clothing color and carried objects provide useful fine-grained matching cues. However, directly using attribute-prompt similarities as retrieval scores is unreliable because templated prompts can be biased, correlated, and poorly calibrated. We therefore propose \textbf{PR}ompt-guided \textbf{A}ttribute-based \textbf{I}nference-time \textbf{S}imilarity \textbf{E}nhancement (\texttt{PRAISE}), a query-side test-time adaptation framework for VLM-based ReID. Instead of treating prompts as scores, \texttt{PRAISE} uses a transferable attribute-prompt bank to construct an \emph{Attribute-Conditioned Subspace} (ACS): a query-dependent low-rank subspace induced by the most compatible attribute prompts. ACS provides a controlled attribute-conditioned geometry for reshaping query--gallery affinities and guiding lightweight inference-time updates without identity or attribute labels. Experiments across multiple I2I- and T2I-ReID benchmarks show that \texttt{PRAISE} consistently improves VLM retrieval backbones and outperforms both generic and ReID-specific TTA baselines.

Tue 20 OctComputer Vision and Pattern Recognition
The gist
Person re-identification is the task of recognizing the same person from different images or descriptions, which is hard when lighting, viewpoint, or background changes. The authors note that using attributes like clothing color can help, but directly comparing attribute matches is not reliable. They propose PRAISE, a method that adapts the search space based on the most relevant attributes for each query, improving matching without needing extra labels. This approach works for both image-to-image and text-to-image searches and shows better results in experiments.
Open → 2610.27944v1

Vision language models struggle to match attributes with objects in images

Stuck on Shuffle: On Attribute-Object Binding in Vision-Language Models

Abstract: Retrieval is a fundamental task in multimodal systems: given an image or text query, the goal is to find the most relevant images or texts in a collection, often using a shared embedding space. Reliable retrieval requires these embeddings to capture not only which concepts are present, but also how they are composed. One key compositional challenge is attribute--object binding: encoding which attributes belong to which objects. Existing benchmarks provide a limited view of this capability, typically covering a subset of modality pairings, simple two-object swaps, or small attribute sets. Consequently, it remains unclear how robustly current models preserve attribute--object bindings across different modality pairings and more complex settings. We therefore introduce SABER, a controlled benchmark for evaluating attribute--object binding across all four image and text modality pairings, spanning diverse attribute types and scenes with up to ten objects. Evaluating models on SABER reveals substantial binding failures in both CLIP-style dual encoders and recent MLLM-based embedders. These findings motivate us to ask how binding can be improved efficiently. While multimodal supervision may be effective, collecting suitable images is difficult and costly. Text, on the other hand, is well suited to binding supervision: it is cheap to generate, supports a broader range of attributes and enables precisely controlled hard negatives. We thus finetune an MLLM-based embedder exclusively on synthetic attribute-binding text. Despite the absence of image supervision, the gains transfer to cross-modal and image--image retrieval while preserving general multimodal embedding performance. Our model achieves SOTA results on SABER while also improving on negation and instance-retrieval benchmarks, demonstrating broader benefits of our binding-focused supervision.

Thu 15 OctComputer Vision and Pattern Recognition
The gist
Finding images or texts that match a description requires understanding not just what things are in the scene, but which details go with which objects. Current models often mix up attributes and objects, like pairing the wrong color with a shape, especially when dealing with complex scenes or different ways of asking. The authors created a new test called SABER to better measure this problem across images and text, finding that many models fail. They improved one model by training it only on specially made text that describes correct attribute-object matches and this helped the model do better even without seeing more images.
Open → 2610.22602v1

Flymark embeds invisible watermarks in 3d gaussian splatting files

FlyMark: Training-Free Invisible Watermarking of 3D Gaussian Splatting via a Fruit Fly Connectome

Abstract: A trained 3D Gaussian Splatting (3DGS) scene ships as a portable parameter array that can be copied, pruned, requantized, or repackaged outside its training pipeline, so ownership evidence is most useful when it lives in the released parameters and remains checkable long after the embedding tooling is gone. Existing 3DGS watermarks typically tie embedding or extraction to scene optimization, a learned decoder, or rendered views, so the evidence survives only as long as a second trained artifact does. FlyMark instead writes a keyed message into the parameters a 3DGS file already stores. Its carrier directions are derived from the photoreceptors of a published connectome, a citable versioned artifact that fixes the geometry exhaustively and leaves nothing to tune per scene. A virtual observer reads cone-wise apparent luminance along a scene-normalized orbit from stored centers, colors, and opacities; a keyed dithered quantization-index-modulation code replicates each message bit across these observations; and one sparse bounded least-squares solve realizes the targets through achromatic shifts of existing degree-zero colors under a hard per-channel linear-RGB bound. All geometry and higher-order appearance parameters are preserved bit-identically, and extraction needs only cone queries, rounding, and majority voting. Under a model-domain threat model on synthetic and real scenes, FlyMark attains high clean bit accuracy and visual fidelity while cleanly separating matched from wrong keys.

Thu 8 OctComputer Vision and Pattern Recognition
The gist
Digital 3D scenes can be copied and shared easily, making it hard to prove who owns them. The authors created FlyMark, a method that hides ownership information directly inside the 3D scene's data without needing additional tools or training. It uses a pattern inspired by a fruit fly's eye structure to encode secret messages invisible to viewers but recoverable later. This watermark doesn’t change the scene’s appearance or shape and can be checked even after the original tools are lost.
Open → 2610.11364v1

Vision transformers improve image search with region-aware tokens

Region-Aware CLS Token Augmentation for Fine-Grained Image Retrieval

Abstract: Image retrieval methods often rely on a single global semantic descriptor extracted from an image, e.g., the [CLS] token in vision transformers. However, trying to squeeze all the semantic information of an image into a single descriptor can hurt downstream retrieval performance, especially for fine-grained retrieval tasks. In this work, we augment the semantic tokens in the newer visual transformers, the global [CLS] token and the four register tokens, with a carefully selected collection of spatial tokens, aiming to capture the spatial region representation that characterizes the contents captured in each of the semantic tokens. We leverage the DINOv2-reg model, which includes register tokens that emergently learn object and part-based representations. For each "cue" token ([CLS] and each register token), we find a "buddy" image patch token and extract an N x N patch region to produce a set of localized ROI tokens. Our approach automatically captures important regions of interest without any external bounding boxes or saliency modules, purely by matching semantic tokens with their spatial representation regions. Furthermore, we incorporate these tokens into a multi-vector retrieval framework inspired by ColBERT, enabling fine-grained matching via a per-token alignment mechanism while avoiding the large storage cost of keeping all patch embeddings. Through extensive experiments, we find that (1) register tokens encode useful fine-grained details that can complement the [CLS] token; (2) automatically pooled ROI tokens further improve fine-grained discrimination; and (3) multi-vector retrieval with a small set of tokens improves over a DINOv2-reg single-vector baseline while remaining tractable for large-scale search. The code is available at https://github.com/IdhcbIan/Augmenting_CLS_with_ROI_tokens.

Wed 7 OctComputer Vision and Pattern RecognitionMachine Learning
The gist
Images are often summarized into a single code to find similar pictures, but this can miss important details, especially when looking for very similar items. The authors use special tokens from a type of AI model called vision transformers to capture more detailed parts of images automatically. By matching these detailed parts with broader image summaries, their method helps find closer matches without needing extra data about where objects are in pictures. Their work shows that using multiple detailed image codes together helps find fine differences more accurately and efficiently.
Open → 2610.10991v1

Self supervised vision model learns interpretable image parts without labels

DisParQ: Self-Supervised Part Concepts for Interpretable Vision Foundation Models

Abstract: Concept-based vision models represent images through an intermediate layer of human-inspectable concepts, so what a model relies on can be traced to those concepts. However, those models are often limited to fixed categories or depend on language to define their concepts. We introduce DisParQ (Discrete Parts with Quantized attributes), a method that learns spatially grounded, discrete concept representations from a powerful frozen vision-only self-supervised backbone. It requires no class labels and no language supervision. Each image patch is assigned to exactly one concept from a learnable prototype dictionary, and only a sparse subset of concepts may activate per image. To capture how each concept varies across images (e.g., the type of a "wheel"), we learn continuous residuals alongside the concepts and then quantize them into discrete attributes. A spatial decoder reconstructs the backbone's representation from the concepts and attributes alone, so successful reconstruction means that the discrete representation preserves the backbone's information. We evaluate DisParQ across seven datasets, from general recognition (ImageNet, PartImageNet, Places) to fine-grained benchmarks (CUB, Cars, Dogs, Flowers). We show that DisParQ closely matches its frozen DINOv2 teacher on ImageNet linear probing (83.2% top-1), achieves higher concept consistency than language-aligned models, remains competitive on fine-grained recognition, and enables cross-category part-based retrieval.

Wed 7 OctComputer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
The gist
Understanding how AI models see images is tricky because their internal features are often hidden. The authors created DisParQ, a method where the model learns to represent images using meaningful parts and their variations without needing any labels or language help. This lets us see which parts the model uses to recognize things and even find similar parts across different objects. DisParQ works well on a variety of image tasks and matches the performance of top AI models.
Open → 2610.09802v1

Preserve and compose training improves zero shot composed image retrieval

Preserve-and-Compose Training for Composed Image Retrieval

Abstract: Composed image retrieval (CIR) aims to retrieve images that satisfy a user-specified modification while preserving relevant visual content from a reference image. Collecting target images for this purpose is costly, motivating zero-shot CIR methods that instead use target captions as supervision. However, target captions may omit source details that should be preserved. We therefore propose, Preserve-and-Compose Training, which complements target-caption supervision with visual evidence from the source image. PACT learns from image--text--text (ITT) triplets without target images or gallery updates, aligning composed queries with target captions while preserving source evidence through visual supervision. We further introduce Chord scoring, which combines target similarity with source-relative directional agreement in the frozen image space. Results across four ZS-CIR benchmarks show that combining target-caption supervision with source-image evidence leads to strong retrieval performance across datasets, backbone scales, and external galleries. The code is available on https://github.com/sehyunkwon/PACT.

Fri 25 SeptComputer Vision and Pattern Recognition
The gist
Finding images that match a user’s description of how to change a picture is hard, especially when we lack examples of the final images. The authors propose a new training method that uses not only text descriptions of the target images but also visual details from the original picture. This helps the system remember important parts of the source image while applying changes described in text. They also introduce a new way to score matches that balances following the text and keeping the original picture’s look. Their tests show better image search across different settings without needing new example images.
Open → 2609.31202v1