Motion transfer system anchors and routes motions to correct objects

Regional Anchor Routing for Multi-Object Motion Transfer

Computer Vision and Pattern Recognition

Summary

Moving multiple objects in a video by copying their motions from different videos is tricky because motions can get mixed up between objects or need slow tweaking for each new video. The authors developed RART, a system that keeps motions tied to the right objects using special tokens anchored to object regions in the video. This system can combine different motions into one video without extra tuning for each new clip, making the process faster and more accurate.

What this means in practice

  • For video editors: Combine motions from multiple reference videos into a single target scene with object-specific accuracy without extra tuning for each new source video.
  • For game developers: Enable realistic animation of multiple characters or objects by transferring specific motions efficiently from various video sources.

Authors

Yuze Li, Fuchen Long, Xinlong Sun, Cheng Yan, Xinyu Zhang, Xi Chen

Abstract

Compositional motion transfer requires a model to extract motions from multiple reference videos, and apply each motion to a user-specified object in one target scene. Existing methods normally struggle to reliably assign different reference motions to the correct target objects, thus a motion intended for one object may affect another. Meanwhile, some motion-specific tuning approaches are still facing another issue: they must run a new optimization whenever the reference video changes, making repeated transfers inefficient. To address these limitations, we propose RART, a framework that unifies object-wise motion extraction, target-region anchoring, and frame-aligned injection into a frozen video Diffusion Transformer for multi-object motion transfer. For each reference motion, RART extracts temporally ordered features from the masked reference video, and then conditions them on the target first-frame mask to form mask-anchored motion tokens. Frame-aligned motion cross-attention allows each generated frame to read only the motion tokens at the corresponding time slots, composing all object motions in one denoising process. During training, a motion-attention routing loss encourages each target region to attend to its corresponding motion tokens and reduces interference between objects. Therefore, RART can handle novel reference-target combinations without per-video optimization. Experiments show that RART achieves accurate motion preservation, efficient inference, and strong compositional generalization.