Papers for

video model engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Video diffusion transformers prune experts to save memory and improve use

DIET: Deletion-response Expert Trimming for Video Diffusion Transformers

Abstract: Video diffusion transformers (DiTs) increasingly adopt mixture-of-experts (MoE) architectures to reduce active computation, but their full expert storage remains costly. Existing one-shot pruning criteria mainly rely on static activation or routing statistics and cannot capture layer-level re-routing after expert deletion. We introduce DIET, a training-free expert pruning framework based on deletion responses. A single all-expert calibration pass records expert outputs and router states for matched conditional and unconditional tokens. Candidate deletions are then replayed from cached tensors, requiring no additional model forward passes. The resulting deletion-response signatures characterize each expert by the changes induced when it is removed. DIET selects retained experts by minimizing Overall Diversity Loss (ODL), which preserves directional coverage in signature space, and combines intra-layer local search with an inter-layer regression-guided budget search to allocate experts across layers. On LingBot-Video 30B-A3B, pruning 50% of experts (6,144 to 3,072) reduces the checkpoint from 57 GB to 30 GB and enables single-card deployment on a 48 GB GPU without fine-tuning. Under a fixed 284-case VBench protocol, the VBench Total increases from 0.7941 to 0.8115. Across tested retention budgets, DIET consistently outperforms competitive pruning baselines adapted from large language models.

Tue 29 SeptArtificial Intelligence
The gist
Large video diffusion transformer models use many expert components that make them hard to store and run on common GPUs. The authors present DIET, a way to decide which expert components can be removed without retraining, by testing how the model responds when each expert is deleted. This approach shrinks the model size nearly in half while maintaining or even improving performance, letting it run on smaller GPUs. DIET works better than other pruning methods adapted from language models.
Open → 2609.37829v1