Lightweight edge system speeds up 3d human pose view planning
LEAP-NBV: Lightweight Edge Active-Perception for Foundation-Model Next-Best-View Planning
Artificial Intelligence
Summary
Understanding the 3D shape and pose of people is useful for robots doing tasks like rescue or inspection, but the powerful models that do this usually need big computers. The authors created LEAP-NBV, a smaller and faster version that can run directly on a robot's edge device without needing to send data elsewhere. They compressed and optimized a big model for edge hardware, enabling robots to quickly decide the best next viewpoint for seeing a person in real time. This achieves faster processing and lower energy use, while keeping accuracy close to the original big model.
What this means in practice
- •For robotics engineers: Enable mobile robots to run human shape and pose estimation onboard with fast planning of viewpoints under limited computing resources.
- •For security system developers: Support edge devices in security cameras to locally estimate human poses quickly for active monitoring without relying on cloud computation.
Authors
Boxun Hu, Jiawei Ge, Axel Krieger, Peng Wang, Tinoosh Mohsenin
Abstract
Foundation models are endowing autonomous systems with greater intelligence, enabling a more comprehensive understanding of the environment through visual perception. A representative example is Human Mesh Recovery (HMR), which provides useful estimates of a target's 3D pose and shape that can benefit tactical missions. However, the size and power demands of such models make them difficult to run on edge platforms and limit their real-time performance, undermining the requirements of tactical edge deployment - especially for active perception, where a mobile robot must plan its next-best view on-board and cannot offload computation under contested communications. We present LEAP-NBV, a lightweight active-perception framework that runs foundation-model-driven Next-Best-View (NBV) planning on-board an edge device. To this end, we distill a family of large HMR teachers, each into a compact 32M student, with an offline mesh objective, then quantize the vision encoder to FP16 and characterize its on-device accuracy and latency. Within an occlusion-aware active perception loop, we evaluate all configurations on the same held-out benchmark and deploy the end-to-end pipeline on an NVIDIA Jetson Xavier NX, reporting measured on-device latency and energy. Distillation recovers 6-7 mm of Procrustes-aligned mean per-vertex position error (PA-MPVPE) over the undistilled student on the test set. Selecting the edge-optimal compression model brings the HMR engine to ~12 ms at a small accuracy cost and runs the full closed loop at 3.6 FPS and 2.6 J per frame, achieving a 2.0x speedup and 3.0x lower energy than the uncompressed model while nearly matching downstream task quality.