PhysVGGT predicts physical object properties from a single photo
PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image
Computer Vision and Pattern Recognition
Summary
Estimating how things feel or behave physically, like how rough or hard they are, from just a picture is tough. The authors introduce PhysVGGT, a tool that can quickly guess detailed physical properties of objects—like friction and density—from one photo, without the usual heavy calculations. It uses a special kind of AI model to analyze both the shape and look of objects pixel by pixel. They also developed a method to create training data without measuring every object physically, making it easier to teach the model. Their approach is much faster and works well even on new kinds of images.
What this means in practice
- •For robotics engineers: Enable robots to quickly estimate object physical properties from images to improve grasping and manipulation tasks without extra measurements or slow processing.
- •For quality control teams: Automatically assess material physical properties from product images to assist in inspection without requiring direct physical tests.
Authors
Sneha Paul, Guile Wu, Bingbing Liu, Dongfeng Bai
Abstract
Physical properties, such as friction, hardness, stiffness, and density, govern how robots should grasp, manipulate and interact with objects, yet estimating these properties from RGB images remains challenging. Existing methods typically employ per-object reconstruction augmented with physical properties or directly query vision-language models at test time, which results in substantial computational overhead that limits their applicability. In this work, we present PhysVGGT, a feed-forward model that predicts dense maps of friction coefficient, Shore hardness, Young's modulus, and density, together with object-level mass, from a single RGB image in one forward pass. The key idea of PhysVGGT is to formulate physical property estimation as a dense per-pixel prediction problem and employ a visual geometry transformer to extract geometry-aware tokens from the input image followed by a dense prediction branch for estimating local physical properties and a global prediction branch for estimating object-level mass. In addition, we introduce a scalable pseudo-label generation pipeline that enables large-scale weakly supervised training for dense physical property prediction, substantially reducing the need for expensive direct physical measurements. Extensive experiments show that PhysVGGT achieves state-of-the-art performance on the ABO-500 dataset and generalizes effectively to the out-of-distribution NeRF2Physics dataset. Moreover, PhysVGGT eliminates the need for per-object reconstruction and test-time optimization, achieving an inference latency of only 0.13s per image, making it $27\times$ faster than the previous state of the art.