Refroute cuts cost and speeds up many-reference image generation
RefRoute: Decoupling Conditioning Cost from References via Compact Residual Conditioning and Spatial Routing
Computer Vision and Pattern RecognitionMachine Learning
Summary
Creating images that combine many different subjects often slows down because computers must carefully handle lots of detailed visual information. The authors introduce RefRoute, a method that smartly reduces how much visual data the model looks at by using low-resolution tokens plus small extra details from full-sized images. It also directs attention only to relevant regions, avoiding extra work when combining parts. Their tests show this approach generates complex images with many references much faster and with better quality than previous methods.
What this means in practice
- •For graphic designers: Generate images combining many elements quickly for creative projects involving multiple subjects in a scene.
- •For game developers: Create complex in-game visuals by efficiently composing multiple characters or objects without long processing delays.
Authors
Wanning He, Yuyao Zhang, Yu-Wing Tai
Abstract
Multi-reference image generation requires preserving the appearance of multiple subjects while composing them into a coherent scene. However, existing diffusion transformers commonly encode references as dense visual token grids and jointly process them with global attention, making conditioning increasingly expensive as the number and resolution of references grow. We present RefRoute, a framework that addresses both reference representation cost and attention overhead through two complementary mechanisms. Compact residual conditioning combines low-resolution latent tokens with lightweight residual features extracted from full-resolution pixels, reducing reference token counts while retaining fine-grained appearance cues. Condition routing and attention routing align reference tokens with their assigned target regions and restrict cross-reference interactions, while allowing selective reference access beyond region boundaries for scene integration. We further introduce RefRoute-Data for training many-reference generation models and ManyRef100, a benchmark spanning human, object, and mixed compositions with 10-17 references. After many-reference fine-tuning, RefRoute achieves an overall Weighted-Ref-VIEScore of 36.06 on ManyRef100, compared with 8.88 for FLUX.2-Klein-9B. Separate inference-cost evaluations show substantially slower latency growth as the reference count increases: at 16 references, our 50-step and 4-step configurations achieve $18.3\times$ and $14.2\times$ speedups over their corresponding FLUX baselines, respectively. These results establish compact reference representations and spatially routed attention as an effective approach to scalable many-reference image generation.