VideoEditor-MoE scales video editing using mixture of experts
VideoEditor-MoE: Scaling Complex Video Editing with Mixture of Experts
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Editing videos with many complex changes needs computers to understand tricky instructions and do many edits at once, which can be very slow if the computer program is very large and fully active all the time. The authors introduce VideoEditor-MoE, a system that uses a mix of specialized little expert models that activate only when needed, saving time and computing power. Their method grows the model size significantly while keeping most experts working and responding to details in the video and editing noise. This approach helps video editing programs handle complicated tasks more efficiently.
What this means in practice
- •For video production teams: Improve efficiency and quality of complex video edits by using scalable models that only activate needed expert components during editing tasks.
- •For machine learning engineers: Develop more efficient large-scale video editing models by integrating sparse mixture-of-experts architecture with attention routing for multiple editing requests.
Authors
Xiao Cao, Fuchen Long, Yansong Qu, Wen Xiao, Xin Zhang, Xinlong Sun, Xi Chen, Zhiyong Huang
Abstract
Complex video editing requires understanding complex instructions while jointly fulfilling multiple editing requests. Meeting these demands through dense model scaling substantially increases computational cost. We introduce VideoEditor-MoE, the first open-source mixture-of-experts framework for video editing. Beyond sparse MLP-MoE layers, MoE-Attention expands attention capacity through routed key--value experts while preserving global interactions among video tokens. Initialized from a 5B video editing model, VideoEditor-MoE scales to 14B and 56B parameters, with approximately 8B parameters activated per token in both variants. Comparisons with existing methods and two complementary baselines, Explicit-MoE and MLP-MoE, assess its benefits over explicit expert guidance and MLP-only expert architectures. Further analyses demonstrate that all experts remain active and reveal that learned routing responds more strongly to sample content and diffusion noise levels than to human-defined editing categories. Spatial analysis further indicates sensitivity to object-level semantic features. We also identify decoupled weight decay as a strategy for stabilizing expert scaling.