Disentanglement-Based Equivariant Learning for Compositional VQA
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionMachine Learning
AI summaryⓘ
The authors focus on a visual question answering (VQA) problem where models must understand new combinations of known concepts. They propose a new method called DEAL that separates the basic ideas from images and questions using techniques inspired by causality, without needing extra data besides the answers. Their approach also enforces consistent reasoning when inputs change in a systematic way, helping the model handle compositional tasks better. Tests on standard datasets show that DEAL works better than current methods in both visual and language challenges.
Visual Question Answering (VQA)CompositionalityConcept DisentanglementEquivarianceCausalityRe-encodingCLEVR-CoGenTGQA-SGLCompositional ReasoningGround-truth Answers
Authors
Zhou Du, Zhaoquan Yuan, Xiao Wu, Changsheng Xu
Abstract
Compositional visual question answering (VQA) represents a challenging yet fundamental task that requires models to comprehend novel combinations of previously learned concepts. The current methods often overlook the disentanglement of underlying concepts and are restricted in terms of their ability to effectively capture the compositional variation mechanism. Moreover, the state-of-the-art techniques depend on additional clues for training, which is not feasible in real-world VQA scenarios. To address these issues, in this paper, we introduce a novel Disentanglement-based EquivAriant Learning (DEAL) framework for compositional VQA, which is guided exclusively by ground-truth answers. In DEAL, we employ causality-inspired interventions to disentangle concepts derived from visual and textual inputs within a re-encoding framework. Based on the principle of equivariance, we subsequently perform a compositional transformation on the inference input and impose the equivariant constraint on the output to augment the compositional reasoning capacity of the model. Comprehensive experiments conducted on the benchmark CLEVR-CoGenT and GQA-SGL datasets validate the superiority of our proposed DEAL approach over the existing state-of-the-art methods for compositional VQA tasks in both visual and linguistic generalization settings.