Carnator speeds up text to video generation by reusing previous work
Carnator: Fast Text-to-Video Generation with Generation-Native Compatibility-Guided Cross-Request Reuse
Artificial Intelligence
Summary
Generating videos from text descriptions is slow because the process repeats many complicated steps. The paper introduces Carnator, a method that speeds this up by smartly reusing parts of previous video creation steps across different video requests. Unlike past methods that guessed similarity based on rough meanings, Carnator looks inside the model’s thinking process to decide how much it can reuse safely. This careful approach lets it speed up video generation more than before while still keeping the video quality good.
What this means in practice
- •For video platform engineers: Reduce server costs by accelerating text-to-video generation for large-scale video hosting and streaming services.
- •For interactive media developers: Enable faster interactive video content creation by shortening the wait time during user-driven text-to-video generation.
Authors
Xingkun Yin, Xuebin Tang, Mingkun Xu, Hongyang Du
Abstract
Video diffusion transformers produce high-quality videos, yet iterative denoising incurs substantial inference latency, limiting interactive and large-scale serving. Most existing acceleration methods focus on individual requests, thereby restricting efficiency gains to redundancy within a single generation trajectory. Recent cross-request reuse offers an additional source of savings, but existing approaches often infer reusability from coarse semantic similarity. This conflates semantic relatedness with generation-level computational compatibility, so aggressive reuse may accept incompatible historical computation while conservative reuse leaves substantial acceleration unrealized. We present \emph{Carnator}, a cross-request acceleration framework that addresses this challenge by extracting and using generation-native compatibility evidence directly from the model's evolving internal states. Specifically, \emph{Carnator} performs a lightweight early probe to construct an Early Signature from internal diffusion states, assessing reuse validity through risk-aware compatibility decisions. The same evidence characterizes reuse scope by localizing target-specific computation and guiding joint reuse of historical latent trajectories and sparse attention connectivity. Across three text-to-video backbones, Carnator consistently achieves higher cache-hit end-to-end acceleration than the evaluated cross-request baselines despite more selective cache acceptance, reaching up to 2.17$\times$ speedup while maintaining competitive generation quality.