EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Video

EgoPhys is a novel framework that enables the generation of deformable physical digital twins from egocentric RGB video, addressing the challenges of predicting complex dynamics in materials like fabrics. It utilizes a compact codebook to distill per-object inverse-physics solutions, allowing for the prediction of spring stiffness fields for unseen objects without the need for test-time optimization. This approach, validated on a real xArm6 robot, demonstrates significant advancements in reconstruction, future prediction, and zero-shot generalization, making it a valuable tool for practitioners in robotics and computer vision working with deformable objects.