Phys4D: Fine-Grained Physics-Consistent 4D Modeling from Video Diffusion

Phys4D is a new pipeline designed to enhance fine-grained physical consistency in 4D world representations derived from video diffusion models. It employs a three-stage training approach: initial pretraining for geometry and motion representations, physics-grounded supervised fine-tuning with simulation data, and reinforcement learning to address residual physical inconsistencies. This methodology significantly improves spatiotemporal and physical coherence over traditional appearance-driven models, which is crucial for practitioners aiming to create more realistic simulations in AI applications.