Multiple subject video generation uses separate image references
MSR: Multiple Subject Reference for Video Generation
Computer Vision and Pattern Recognition
Summary
Generating videos that include multiple characters or elements from different images is challenging because the system must keep each reference clear and distinct. The authors introduce MSR, a method that assigns each input image its own 'slot' to avoid mixing them up in video creation. This approach uses special encoding and modifications to keep characters and backgrounds separate and recognizable throughout the video. Although the method reduces confusion compared to previous ways, handling similar clothing or complex views remains difficult. The authors also experiment with adding voice cues alongside images without changing the visual settings.
What this means in practice
- •For video content creators: Create videos featuring distinct characters and environments referenced from separate images with reduced confusion between subjects.
- •For game developers: Generate stylized scenes combining multiple character references for immersive game cutscenes or animations.
Authors
Guannan Li, Jiaji Chen, Jingyuan Liao, Yu Geng, Baolan Qiu
Abstract
Conditioning a video generator on multiple images requires preserving appearance while associating each reference with its intended role. We present MSR (Multiple Subject Reference), a slot-aware conditioning scheme for LTX-based video generation. Each reference image is independently encoded as a static clip and represented by a separate latent-token group. A compact Fourier-feature multilayer perceptron adds a numeric slot embedding, while slot-dependent temporal offsets modify the group's rotary coordinates. The reference groups are prepended to noisy target tokens and serve as clean context during target-only flow-matching training. We implement this scheme through low-rank adaptation and release the resulting weights and inference workflows. Qualitative examples demonstrate compositions containing distinct characters and referenced environments in realistic and stylized scenes. Development observations suggest reduced reference confusion relative to an earlier continuous-reference baseline, while similar clothing, complex garments, and viewpoint changes remain challenging. We describe the conditioning mechanism, the retained training configuration, and the observed strengths and limitations of the released system. A supplementary audio-reference experiment adds voice conditioning while keeping the visual parameters frozen.