WanPE improves cinematic prompts for longer text-to-video generation
WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation
Computer Vision and Pattern Recognition
Summary
Generating videos from text prompts is challenging, especially for longer and complex scenes involving multiple shots, camera moves, and lighting. The authors developed WanPE, a large model trained on over a million videos, that enhances text prompts to better capture cinematic details and plans at the shot level. This helps video generators create more coherent and visually engaging videos up to 30 seconds long. Tests show WanPE significantly improves viewer preference compared to original prompts and outperforms other commercial tools.
What this means in practice
- •For video content creators: Enhance detailed text prompts to generate more coherent and cinematic multi-shot videos up to 30 seconds long.
- •For advertising agencies: Use improved prompt enhancement for producing polished video storyboards from client scripts efficiently.$Commercial implications: This paper enables tools that automatically convert scripts into high-quality video sequences, improving production workflows.
Authors
Yubo Zhu, Yawen Shao, Ziyun Dai, Zixun Fang, Kai Zhu, Siyang Sun, Haolan Xue, Chuxin Wang, Tingyu Weng, Jingming Luo, Chen Shi, Lianghua Huang, Yufeng Ai, Yuzheng Wang, Wenyuan Zhang, Yu Shang, Yuxiang Bao, Zoubin Bi, Jie Xiao, Jinbo Xing, Jiaxing Zhao, Chongyang Zhong, Hengjian Chen, Chenwei Xie, Akide Liu, Zhehan Kan, Yu Liu, Wei Zhai, Sheng Zhong, Wei Tong
Abstract
Video generation begins in text space by authoring a cinematic screenplay, then materializes into pixels. As contemporary video generators scale to 30 seconds and faithfully follow complex conditions, the textual prompt largely directs the production, planning how actions, camera trajectories, lighting, and sound unfold across multi-shot sequences. In this paper, we present WanPE, a 397B-parameter prompt enhancement model trained on 1.05M real-world videos to master director-level cinematic planning. WanPE formulates shot-level cinematic plans via video-grounded reverse construction and employs Semantic-Consistency GRPO (SC-GRPO) to faithfully preserve user requirements across shots and over time. To benchmark this capability, we curate WanPEval, a human-annotated testbed covering durations from 5 to 30 seconds across varying intent granularities, supported by approximately 11K blind pairwise assessments. When powering Wan3.0's video generator, WanPE-397B boosts human preference over raw user prompts by 10.66-18.84 points at 5-15 seconds and by a dramatic 50.86 points in the 30-second arena. Ablation studies show that reverse construction demonstrates clear superiority over forward rewriting, while SC-GRPO robustly preserves semantic fidelity across model scales. Ultimately, WanPE leads all evaluated commercial offerings at 5-15 seconds and remains competitive with Seedance 2.5 at 30 seconds.