PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
2026-07-11 • Machine Learning
Machine LearningArtificial IntelligenceComputer Vision and Pattern Recognition
AI summaryⓘ
The authors found that video-language models (VLMs) struggle to understand if physical events in videos make sense, like how objects interact or follow basic physics rules. To help with this, they created PhysMRV, a system that uses a special memory made from training videos, organized in three ways: describing scenes, showing how objects interact, and summarizing physics rules. When looking at new videos, PhysMRV finds similar memories and uses them to check if what happens is physically possible, without changing the VLM itself. Tests on different benchmarks showed that PhysMRV improves VLMs' ability to reason about physics without extra training.
Video-language modelsPhysical plausibility reasoningPhysical commonsenseHierarchical memory bankPhysical-event graphsPhysics-rule summariesInferenceBenchmark datasetsVisual question answeringTraining-free verification
Authors
Wenyuan Wang, Lianyu Hu, Hao Wang, Yang Liu
Abstract
Video-language models (VLMs) have achieved remarkable performance on video understanding and visual question answering, yet they remain unreliable in reasoning about physical plausibility, where understanding object interactions, causal dynamics, and fundamental physical principles is essential. This limitation is particularly evident on challenging physical reasoning benchmarks, revealing a persistent gap in physical commonsense reasoning. To address this challenge, we propose PhysMRV, a training-free physical memory and verification framework for physical plausibility reasoning. Unlike retrieval-augmented VLMs that retrieve semantically similar videos as additional context, PhysMRV transforms training videos into a Hierarchical Memory Bank of structured physical knowledge comprising three complementary levels: scene descriptions capturing visual context, physical-event graphs modeling object interactions and causal structure, and physics-rule summaries distilling reusable physical principles and cues. During inference, PhysMRV retrieves physically relevant memories and leverages their structured physical evidence to guide a frozen VLM in verifying physical plausibility, requiring neither fine-tuning nor parameter updates. We evaluate PhysMRV on three challenging physical reasoning benchmarks, ImplausiBench, IntPhys2, and GRASP Level 2, across multiple state-of-the-art VLMs. Experimental results demonstrate consistent improvements over direct prompting across diverse VLMs and evaluation benchmarks, showing that structured physical memories provide an effective and scalable means of enhancing physical plausibility reasoning without additional training.