AI summaryⓘ
The authors study mistakes made by multimodal large language models (MLLMs) when they create image captions, focusing on repeated errors tied to certain visual features, which they call systematic misalignments. They developed Symbal, a tool that uses existing AI models to find and explain these recurring errors in image-caption pairs. To test this, they also created SymbalBench, a large benchmark with millions of images and captions labeled for these errors. Symbal performed much better than previous methods and can help audit captions without needing to look inside the original language models. This work aims to improve how we check AI-generated image descriptions for consistent mistakes.
Multimodal Large Language ModelsImage CaptioningSystematic MisalignmentVision-Language DatasetSymbalSymbalBenchBenchmarkError DetectionAuditFoundation Models
Authors
Maya Varma, Jean-Benoit Delbrouck, Sophie Ostmeier, Akshay Chaudhari, Curtis Langlotz
Abstract
Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image-text pairs. Our work focuses on a class of captioning errors that we refer to as systematic misalignments, where a recurring error in MLLM-generated captions is closely associated with the presence of a specific visual feature in the paired image. Given a vision-language dataset with MLLM-generated captions, our aim in this work is to detect such errors, a task we refer to as systematic misalignment detection. As our first key contribution, we present Symbal, which utilizes a structured, dual-stage setup with off-the-shelf foundation models to identify systematic misalignments and summarize results in natural language. As our second key contribution, we introduce SymbalBench, a benchmark designed to evaluate automated methods on our proposed task. SymbalBench consists of 1.7 million image-text pairs from two domains (natural and medical images), organized into 420 vision-language datasets with annotated systematic misalignments. Symbal exhibits strong performance on this benchmark, correctly identifying systematic misalignments in 63.8% of datasets, a nearly 4x improvement over the closest baseline. We supplement our evaluations on SymbalBench with real-world evaluations, showing that (1) Symbal can accurately surface systematic misalignments in captions generated by four MLLMs and (2) Symbal is a powerful tool for auditing off-the-shelf image-caption datasets. Ultimately, our novel task, method, and benchmark can aid users with auditing MLLM-generated captions and identifying critical errors, without requiring access to the underlying MLLM. Code is available at https://github.com/Stanford-AIMI/Symbal.