Multi-agent system improves consistent video generation across shots
MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimization
Computer Vision and Pattern Recognition
Summary
Creating multi-shot videos with the same characters and settings is hard because each shot might look different or disconnected. The authors developed MVAgent, a system where multiple agents work together to keep characters, camera views, and actions consistent between shots. One agent picks the right camera view for each shot, another keeps track of what happened to keep the story smooth, and a special trainer helps the whole system learn by comparing different video versions. This approach leads to more consistent and well-planned videos compared to previous methods.
What this means in practice
- •For video content creators: Generate multi-shot videos with consistent characters and camera layouts automatically for storytelling and entertainment purposes.$Commercial implications: Enables automated creation of narrative video content with consistent visual elements, useful for studios and online media producers.
- •For game developers: Produce coherent in-game cutscenes with stable character appearance and camera positioning across multiple shots without manual intervention.
Authors
Xiangyu Kong, Wenjie Zhou, Fengping Tian, Lihua Fang, Haoqin Sun, Chenyang Lyu, Longyue Wang, Weihua Luo
Abstract
Multi-shot agentic video generation requires consistent character appearance, stable spatial layout across camera angles, and continuous character state between shots. When every shot is a separate request to a frozen generator, repeated text does not determine appearance, layout or state. We therefore recast the problem as condition construction and present MVAgent, a multi-agent pipeline whose agents collaborate through typed conditioning inputs. Because an environment image shows one viewpoint, a Spatial Grounding agent samples views from generated camera-traversal clips and anchors each shot to the view matching its framing. As generated shots drift from the plan, an Observer records how each shot ends in a continuity memory, from which a Transition agent builds character action and spatial references for the next shot. An Orchestrator composes these inputs into each request. Since a request reveals its effect only after rendering, we train it by agentic reinforcement learning with Trunk-GDPO, which compares rendered candidates at every shot rather than once per video and continues the best as the trunk. With generator and judges frozen, MVAgent attains the highest cross-shot consistency and narrative-planning quality among the compared methods on ViMax-Bench and is preferred over the strongest agentic baseline in human evaluation.