PhysMamba simulates articulated bodies faster with learned models
PhysMamba: Selective State Space Models as Learned Articulated Body Simulators
Computer Vision and Pattern Recognition
Summary
Simulating how articulated bodies move is important but can be slow or require complex inputs. The authors developed PhysMamba, a system that predicts full-body movements from positions and joint actions without needing velocity data. They tested different training methods and found one that gives good accuracy while running very fast on powerful GPUs. This could help improve video-based body tracking by providing a fast, differentiable physics model.
What this means in practice
- •For computer graphics developers: Integrate PhysMamba to accelerate full-body articulated simulations for more realistic animations at near real-time speeds on GPUs.
- •For augmented reality engineers: Use PhysMamba as a fast physics module to improve video-based human body mesh recovery with differentiable simulation.
Authors
Haochuan Zhang, Sinisa Todorovic
Abstract
We introduce the first learned articulated body simulator based on a selective state space model (SSM), called PhysMamba. PhysMamba predicts next-frame full-body state from position, rotation, and joint-action history, without velocity inputs. We compare four architectures under partial- and full-observation inputs and three training protocols. The from-scratch rollout training protocol gives Mamba2 strong short- and mid-horizon accuracy under partial observation (s10 = 43 mm, 2/50 diverged), while the two-stage teacher-based rollout protocol stabilizes GRU but fails for Mamba2. With CUDA graph compilation, Mamba2 reaches 0.107 ms per frame (9,334 FPS) on an H100 GPU, within 1.1$\times$ of GRU's un-compiled throughput, adding under 1% latency to a 30 Hz HMR pipeline and enabling integration as a differentiable physics module for video-based mesh recovery.