EcoVideo: Entropy-Orchestrated Video Generation Paradigm in Cloud-Edge Dynamics
2026-06-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors present EcoVideo, a system designed to make a certain type of video generation faster and more efficient. Instead of treating all video frames the same, their method picks important keyframes to process using a large model in the cloud, while a smaller model at the edge fills in the gaps by estimating motion. They also adjust how many keyframes to use and how much detail to refine based on available internet speed and computing power. Their experiments show this approach speeds up video generation significantly, especially when resources are limited.
DiT video generationlatencyself-attention entropykeyframe selectioncloud-edge computingmotion-aware interpolationtemporal stabilitybandwidth adaptationcompute-limited environmentsvideo denoising
Authors
Jiayu Chen, Hengyi Zhang, Maoliang Li, Minyu Li, Zihao Zheng, Xuanzhe Liu, Guojie Luo, Xiang Chen
Abstract
DiT video generation is latency-intensive due to iterative full-frame denoising, while prior cloud-edge methods largely rely on static inter-step decoupling and cannot leverage inter-frame similarity or adapt to system dynamics. We propose EcoVideo, an entropy-orchestrated framework for dynamic inter-frame decoupling: early-stage self-attention entropy provides a training-free estimate of frame-wise information density for frame selection; a cloud large model denoises sparse high-entropy keyframes; and an edge lightweight model reconstructs the remaining frames via motion-aware interpolation with refinement for temporal stability. EcoVideo further adapts the keyframe budget and edge refinement depth to real-time bandwidth and compute availability, optimizing end-to-end latency under constraints. Experiments on representative DiT video generators show improved quality--efficiency trade-offs and up to 2.9x end-to-end speedup in low-bandwidth, compute-limited edge settings. Code is available at https://github.com/IF-LAB-PKU/EcoVideo.