PhysReal couples 3DGS-based appearance with differentiable MPM physics to recover object-specific deformable dynamics from single-view interactive videos. By modeling spatially varying hybrid expert-neural constitutive fields, PhysReal combines interpretable physical priors with neural residuals, enabling accurate reconstruction, future prediction, and downstream robotic applications.
Learning physically plausible dynamics from visual observations is a fundamental capability for interactive world models and embodied agents. However, real-world deformable objects are particularly challenging in this setting, as their behaviors often arise from heterogeneous and complex physical responses. To tackle this challenge, we introduce PhysReal, a novel video-driven deformable object modeling and simulation framework. Specifically, we develop a spatially varying hybrid expert-neural constitutive model integrated with the differentiable MPM and 3DGS, where expert models provide interpretable physical priors and neural residuals capture responses beyond analytical formulations. The constitutive field is represented by spatially distributed patches, allowing continuous modeling of local material variations. We further design a progressive curriculum learning strategy that gradually learns constitutive behaviors from global properties to local variations and residual responses, together with a unified motion-mask supervision scheme for learning from sparse visual observations. Extensive experiments on diverse deformable object interactions demonstrate that PhysReal achieves superior performance in dynamic reconstruction and future state prediction, while showing strong potential for downstream robotic applications.
PhysReal consists of three key components. Interactive Video Parsing extracts temporally consistent object masks, motion trajectories, and an object-centric 3D representation from single-view interaction videos. Differentiable Physics Simulation couples MPM with a spatially varying hybrid expert-neural constitutive model to capture heterogeneous material responses. Finally, a progressive curriculum jointly leverages motion and mask supervision to optimize global material properties, local constitutive variations, and neural residual responses.
Qualitative results across the interaction sequences used in our evaluation.
We compare PhysReal with four representative methods: GS-Dynamics, Spring-Gaus, PhysFlow, and PhysTwin. For a fair comparison, all methods are evaluated under the same single-view observation setting.