A Plug-and-Play 2D Motion Interface for Real-World Motion Language Models
2026-08-17 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem that 3D motion data is hard to get from single-camera videos, which makes it tough for Motion Language Models (MoLMs) to work well in real life. They created a simple add-on called the 2D Motion Interface that lets existing MoLMs understand 2D motion inputs without needing to change or retrain the models. Tests show their method works about as well as using 3D data and is better than training models only on 2D inputs. They also made a new dataset from real videos and showed 2D data can be more useful than 3D data when using current 2D-to-3D pose estimation. This suggests their 2D interface is a practical way to use MoLMs with normal video footage.
Motion Language Models3D motion data2D motion interfacemonocular videopose estimationtokenizationreal-world datasetadaptermodel fine-tuningpose representation
Authors
Kaname Yokoyama, Norimichi Ukita
Abstract
Motion Language Models (MoLMs) typically understand human motions by tokenizing 3D motion and processing the resulting tokens using a language model. However, obtaining accurate 3D motions from monocular videos is challenging, limiting their real-world applicability. To address this issue, we introduce a plug-and-play 2D Motion Interface that enables 3D-pretrained MoLMs to accept 2D motion inputs without modifying or fine-tuning the original models. Experiments on public datasets show that our method achieves performance comparable to 3D motion inputs across multiple MoLMs and outperforms training MoLMs from scratch on 2D motions. We further construct a monocular real-world video motion evaluation dataset and introduce a real-video adapter, demonstrating the usefulness of 2D motions over 3D motions under the evaluated monocular pose-estimation setting. These results suggest that 2D motion provides a practical interface for deploying MoLMs in real-world motion understanding settings. Code is available at https://github.com/irajisamurai/2D-Motion-Interface.