Unified model improves image reconstruction and generation quality
From Feed-Forward to Flow: Unifying Reconstruction and Generation Is Easier Than You Think
Computer Vision and Pattern Recognition
Summary
Reconstructing images from partial views tends to blur ambiguous areas, while generating new details can create inconsistent scene parts. The authors propose a single method that does both using a flow-based model with one predictor. With one step, it behaves like traditional methods, averaging ambiguity into blur for consistency. With multiple steps, it adds detailed, plausible content where the images lack information, adjusting detail quality based on available context. This approach works well for 3D appearance and geometry tasks, producing clearer reconstructed images and surfaces with more flexible computation at test time.
What this means in practice
- •For computer vision engineers: Improve 3D scene reconstruction and novel view synthesis with a unified model that flexibly balances detail consistency and generation.
- •For video game developers: Generate consistent and detailed 3D environments from partial data, reducing visual artifacts like flying pixels and veil effects during rendering.
Authors
Haoru Wang, Qianfan Shen, Kai Ye, Wenzheng Chen, Baoquan Chen
Abstract
Reconstruct where the images provide evidence, and generate where they do not: recent success of spatial world models such as Atlas (World Labs Team, 2026) highlights the value of unifying reconstruction and generation in one model. Yet the two have long lived in separate paradigms with distinctive failure modes: feed-forward reconstruction averages ambiguity into blur, while conditional generation invents plausible but scene-inconsistent detail. In this work, we present a unified flow-based formulation for reconstruction and generation, where a shared clean-target predictor performs direct reconstruction at its single-step endpoint and unfolds conditional generation through multi-step flow. A controlled toy study reveals the mechanism: with a single step, the predictor collapses to the conditional mean just like feed-forward methods, favoring consistency over diversity. With multi-step inference, the fidelity of generated details grows with context richness: closer observations reduce ambiguity and yield better-matched details. We further instantiate the formulation in appearance and geometry 3D tasks. JiT-LVSM improves perceptual and distributional quality in novel view synthesis, while JUSt3R retains competitive single-step geometry prediction with additional multi-step inference capabilities that reduces veil and flying-pixel artifacts, producing cleaner surface structure with greater test-time compute. Together, they show that reconstruction and generation can share both a formulation and a backbone, with their behavior governed by denoising configuration---making unification surprisingly simple.