Papers for

visual content localizers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

ImgMT dataset benchmarks text translation inside images across 129 languages

ImgMT: Wikipedia-based Dataset For Evaluating Text-in-Image Translation

Abstract: Text-in-image translation aims to translate all textual content embedded in an image. In Text Image Translation (TIT), the output is translated text; In-Image Machine Translation (IIMT) additionally renders the translation back into the image. We introduce ImgMT, a massively multilingual dataset and benchmark mined from localized SVG illustrations on Wikimedia Commons, focusing on diagrams, maps, and infographics. ImgMT contains 35,612 aligned image pairs grouped into 1,855 image sets, spanning 129 languages, 20+ writing systems, and 4,886 language pairs. We propose a multi-level evaluation protocol that measures source-side text-region detection and OCR accuracy alongside target-side translation quality. We benchmark several multimodal large language models and a dedicated TIT system in English->X and X->English settings, and find that the source language and writing system substantially affect TIT quality. Finally, we show that ImgMT is not well suited for isolating the contribution of visual information in multimodal translation.

Mon 26 OctComputation and Language
The gist
Translating text embedded in images like diagrams or maps is tricky because you have to find the text, read it, and then translate it. The authors created ImgMT, a large dataset of images with text in 129 languages from Wikipedia. They also set up ways to measure how well systems detect, read, and translate this text back into images. Their tests show that the language and script used affect translation success, and they found that images don’t add much extra help in understanding the text for translation.
Open → 2610.35416v1