Self-improving program director for explainer videos improves video scripts
Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos
Artificial IntelligenceGraphicsMultimediaSoftware Engineering
Summary
Explainer videos are usually just sequences of pixels, making them hard to analyze or improve automatically. This paper treats these videos as programs that generate frames using deterministic rendering. The authors built a system called Reels-RSI that improves these video programs by checking their code, learning rules from past fixes, and applying those rules to catch errors early. Human judges decide which program improvements to keep. Their experiments showed the system can detect and prevent defects in the video programs over time.
What this means in practice
- •For software engineers: Improve reliability of video generation pipelines by automatically detecting and preventing programming errors in animation scripts before rendering.
- •For content production teams: Ensure higher quality and fewer revisions for explainer videos by applying automated program-level fixes and checks during video script development.
Authors
Wei Lin
Abstract
Explainer videos -- a worked geometry problem, why the sky is blue, how a dynasty fell -- are usually one more target for pixel video generation. We argue they are a different kind of artifact: a program (script, storyboard and per-frame animation code) that a deterministic renderer turns into frames. A program can be read, diffed, tested and audited; pixels can only be watched. Reels-RSI is an open-source reel director for any agent harness; its self-improvement happens at the program level rather than in weights or prompts. Every revision pass is logged with a code snapshot; a retrospective pairs each check finding with the diff that fixed it and proposes lessons; a lesson that a static check can catch is compiled into a rule with a must-fail and a must-pass example, so the memory is unit-tested; and a rule learned today runs over everything made before it. A bounded, benchmark-gated step lets an agent rewrite the director's playbook, kept only when blind pairwise judgments on held-out topics prefer it; nothing enters memory without a human decision. Across 111 reels -- a field study, two playbook evolutions, a 2x2 memory ablation, a weaker director, a 48-reel learning curve and a judge-reliability study -- we find that the rule learned in the field caught four latent defects in frames authored before it existed and none after; that rules are an audit for a strong director (eight unguarded reels, no violation) and a guard for a weaker one (thirty violations unguarded versus five); that lessons shorten revision loops but do not compound or move the judge; and that the first rewrite's 11--1 verdict was a position bias in our vote design (judges pick the second-shown reel 62--81% of the time), masking scripts that traded thirteen facts for craft. We document four failure modes of the preference signal, which check caught each, and the fixes. Code, rules, lessons and data are public.