Papers for

animation studio teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language models improve limb motion captions with graded error scoring

Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

Abstract: Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, particularly across camera cuts. Improving these details requires evaluation and training that distinguish missing information from incorrect assertions. We introduce FlexBench, a benchmark spanning 3,105 shots and 18,161 evaluation queries, with human-verified identities and systematic per-person coverage of fine-grained limb actions and states. Its reference-derived checklists support automated assessment of complete captions in their person and shot contexts. Our Graded Physical Alignment score (GPA) awards credit for correct content and deducts points for incorrect or fabricated actions, making these errors explicit in the aggregate score. Building on this rubric, we propose Graded Margin Direct Preference Optimization (GM-DPO), which assigns stronger preference margins and greater training weight to more severe action errors. Across three VLM backbones, GM-DPO achieves the highest substantive-action and GPA scores among the evaluated preference objectives, improving GPA over DPO by 2.02-3.40 points. On Qwen3-8B, it reduces the weighted hallucination rate by 21.3% relative to DPO. These gains accompany sustained long-form output, improved shot structure, and competitive performance on three additional multimodal benchmarks.

Tue 29 SeptComputer Vision and Pattern Recognition
The gist
Video captioning models sometimes get confused about who is doing what action, especially involving specific body parts and when scenes change. The authors created a new dataset called FlexBench to better evaluate this by checking precise limb movements per person. They also developed a scoring system that rewards partial correctness and penalizes wrong guesses more clearly. Using this system, they introduced a training method that better teaches models to avoid serious mistakes, leading to more accurate video descriptions about who does what with which limbs.
Open → 2609.36628v1