Summary
As more data comes in many forms like text, tables, and images, it becomes harder for computers to answer questions using all this mixed information. The paper looks at three main approaches to this problem, showing how systems have changed from handling each type of data separately to using one unified language model that understands everything together. The authors find that treating all data as text and using large pre-trained language models helps computers answer questions more accurately and reliably. However, some challenges remain, such as losing details when converting data and mistakes that happen in multi-step processes. This study helps us understand how future systems might better connect different types of information for smarter answers.
multimodal dataquestion answeringcross-modal interactionspre-trained language modelsmodality transformationpipeline error propagationexact matchF1-scoreunified architecturesinformation loss
Abstract
The rapid growth of multimodal data has intensified the need for question answering (QA) systems capable of reasoning across heterogeneous sources such as text, tables, and images. In this paper, we present a comprehensive methodological comparison of three influential frameworks, namely Multimodal Adaptive Extraction (MAE), Solar, and UniMMQA, tracing the evolution of multimodal question answering from modality-adaptive pipelines to fully unified architectures. We examine how each approach models cross-modal interactions, transforms heterogeneous inputs, and performs reasoning, highlighting key design differences in modality representation, reasoning, and answer generation. Our analysis demonstrates a clear shift from explicit modality-specific processing toward unified text-centric formulations enabled by pre-trained language models (PLMs). Empirical comparisons across benchmark datasets show that this transition leads to substantial improvements in both Exact Match (EM) and F1-Scores, with UniMMQA achieving the most consistent and scalable performance. Despite these advances, we identify persistent challenges, including information loss during modality transformation, error propagation in multi-stage pipelines, and limitations in capturing fine-grained cross-modal dependencies. Overall, this study provides a deeper understanding of current design trends and offers insights into the future direction of unified multimodal reasoning systems.