Sensorimotor World Models: Perception for Action via Inverse Dynamics

The article introduces a sensorimotor world model (SMWM) that utilizes inverse dynamics regularization for end-to-end training, addressing the challenges of representation collapse and action relevance in latent world models. By enforcing latent states to capture action-related information, SMWM effectively learns compact, interpretable latent spaces and demonstrates competitive planning performance in various 2D and 3D control tasks. This approach is significant for practitioners as it allows for stable model training from offline, reward-free trajectories, enhancing the ability to develop action-oriented AI systems.