Tide speeds up text to video prompt improvements without slow searches
TIDE: Trajectory-Informed Skill Distillation for Efficient Text-to-Video Prompt Optimization
Computer Vision and Pattern Recognition
Summary
Text-to-video models need very specific text prompts to make good videos, but finding the best prompt usually takes a lot of trial and error. The authors developed TIDE, a method that learns how to rewrite prompts quickly by creating a reusable set of rules that adjust the text in smart ways. Once trained, TIDE can improve prompts efficiently with just one video generation instead of many, saving time and computing power. It also works well on new video models it wasn't trained on, matching or beating other more expensive methods.
What this means in practice
- •For video content creators: Improve text prompts efficiently to generate higher quality videos with fewer trial runs, reducing wait time and compute resources.
- •For ai product developers: Integrate a reusable prompt optimization skill that supports multiple text-to-video models for scalable video generation services.$Commercial implications: Enables video platforms to offer faster and more accurate prompt improvements as a premium feature using TIDE’s distillation method.
Authors
Chengxing Zhou, Yuliang Huang, Jingwei Song
Abstract
Text-to-video (T2V) models are highly sensitive to prompt specificity, while iterative prompt optimization typically improves generation quality only through repeated video generation and vision-language model evaluation at inference time. We propose TIDE, a framework that distills this costly search process into an interpretable and reusable prompt-rewriting Skill. TIDE formulates prompt rewriting as a set of source-conditioned rule modules, each targeting a distinct semantic dimension. Once distilled, the frozen Skill activates the relevant rules and generates both positive and negative prompts in a single LLM call, followed by only one T2V generation, without test-time visual search or backbone fine-tuning. Evolved using only 100 prompts over 14 hours, TIDE achieves prompt-optimization performance comparable to computationally expensive search-based methods. Moreover, when transferred to previously unseen video-generation models, TIDE outperforms existing approaches specifically designed or trained for those models. Code is available at https://github.com/xiningin/TIDE_T2V.