Human motion and camera paths generated with translation control
AESOP: Asymmetric Human-Camera Generation with Translation-Intensity Control
Computer Vision and Pattern Recognition
Summary
Videos show both what is happening and how it is filmed. This paper presents a new way to create videos by generating human movements first, then controlling the camera movement to show that action effectively. The system can also adjust how much the camera moves towards or away from the subject, making the video framing better. The researchers tested this method on a collection of videos and found it improves how natural and varied the camera movements look.
What this means in practice
- •For video content creators: Create videos with realistic action and camera movements that can be adjusted for camera motion intensity.
- •For game developers: Generate human animations and adaptive camera paths that respond to player or character actions with controllable camera movement.
Authors
Jingzhong Lin, Zhanke Wang, Heng Li, Wenxiang Liu, Zhao Zhang, Kecheng Tang, Dongdong Xiang, Changbo Wang, Di Kang, Chunchao Guo, Linchao Bao, Gaoqi He
Abstract
Human motion defines an action, while a camera trajectory determines how it is presented. Camera generation for a given human motion and joint human-camera generation are usually treated as separate tasks, although both share an asymmetric dependency: human motion can be generated independently, whereas the camera responds to the realized action. We introduce AESOP, a unified framework with an independent human pathway and a shared human-conditioned camera generator. Its asymmetric architecture serves both tasks while preserving the human output during camera generation. Although human context anchors the shot to the action and camera text describes its movement, translation intensity remains underspecified. We therefore construct trajectory pairs that differ in camera translation magnitude while sharing human motion and camera text, then use these pairs to learn an explicit intensity condition. Experiments on the PulpMotion dataset demonstrate strong camera distributional and framing quality in both tasks and effective control over camera translation intensity.