Hybrid model improves efficiency and quality in multimodal reasoning
Beyond the Sequentiality Bottleneck: A Hybrid Diffusion-Autoregressive Framework for Efficient Multimodal Reasoning
Artificial Intelligence
Summary
Autoregressive models think step-by-step but can be slow because they handle tasks in sequence. Diffusion models work faster by doing many things at once but sometimes lose track of meaning. The authors designed a system called CoRAL that combines both: one part quickly generates multiple possible solutions in parallel, and another part picks the best and final answer carefully. Their method performed better than other diffusion-based approaches and was as good as strong step-by-step methods, while also being faster.
What this means in practice
- •For multimodal ai developers: Enable more efficient and higher-quality reasoning by combining parallel plan generation with sequential answer synthesis in vision-language tasks.
- •For natural language processing engineers: Improve response speed and coherence in complex reasoning systems that integrate text and images by adopting hybrid diffusion-autoregressive frameworks.
Authors
Wei Cai, Jian Zhao, Yuchen Yuan, Xuelong Li
Abstract
Autoregressive models (ARMs) face a "Sequentiality Bottleneck" in reasoning, while diffusion language models (DLMs) offer parallelism but lack semantic coherence. To addresses both limitations, we propose Collaborative Reasoning Asymmetric Learning (CoRAL), a hybrid framework via an Asymmetric Planner-Executor (APE) architecture: a DLM Planner generates K diverse reasoning trajectories in parallel, while a large ARM Executor selects the optimal plan and synthesizes the final answer. To train the Planner effectively, we introduce Reward-Guided Denoising for CoT (RDC), which integrates outcome-based rewards into the diffusion process to steer it toward goal-directed reasoning. Experiments on vision-language benchmarks show that CoRAL significantly outperforms prior diffusion-based methods and achieves competitive performance against strong ARMs, achieving both superior efficiency and quality.