Short video content labeling improves with autopilot framework
Towards Autopilot Labeling for Short Video Content Governance
Computation and Language
Summary
Short videos are growing very fast, but it's hard and expensive to label their content manually to decide what is okay or not. The authors designed a step-by-step automatic system that gets better over time, starting from basic computer learning models and going to advanced ones that can work on new kinds of videos without training. At the highest level, their system can even handle tricky cases on its own, needing less help from humans. Tests show that each step improves how accurately videos are labeled and reduces the amount of human work by a lot.
What this means in practice
- •For content moderation teams: Automate short video labeling to speed up detection of inappropriate content and reduce reliance on manual review.
- •For social media platform developers: Integrate adaptive labeling agents to handle novel or complex video content types with less human intervention.
- •For online marketing teams: Use autopilot labeling technology to efficiently classify user-generated short videos for targeted advertising.$Commercial implications: Enables services that provide automated video content classification to marketers for more precise ad placements.
Authors
Junwen Chen, Hongwei Wang, Zixuan Wang, Jingbing Wang, Gorden Liu, Sheng Siong Ong, Xiaojiao Zang, JuChin Chao, Ze Jin, Kaiwen Men, Yin Li, Jeremy Li, Zhuolin Hao, Hongyu Xiong, Yang Song
Abstract
The explosive growth of short videos has presented significant challenges to content governance. Traditional manual labeling is inefficient, slow to iterate, and costly. Therefore, we explore and develop a multi-stage autopilot labeling framework aimed at progressively increasing the level of automation in short video content annotation. To move beyond the heavy reliance on human expert labeling at the initial L0 stage, we first introduce a discriminative MLLM labeler at the L1 stage, which is fine-tuned with supervised data to directly predict labels for short videos. Advancing to the L2 stage, we shift to a domain-adaptive MLLM pretrained on short video data and use it for zero-shot labeling, enabling flexible generalization without task-specific supervision. In the L3 stage, we propose an agentic annotation workflow, and equip the agent with several key tools, which empower the agent to handle complex, unfamiliar, or vaguely defined cases, significantly reducing the need for human intervention. We conduct extensive experiments across various content issues, and results demonstrate that each stage of the autopilot labeling framework progressively improves labeling accuracy, and has significantly reduced manual labor costs in real-world applications by up to 85%.