Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models

The paper presents the grounded latent-action world model (GLAM), which enables imitation learning from heterogeneous data sources by creating a shared latent action space grounded in future observation predictions. This approach allows for effective action transfer across different data sources, improving the stability and generalization of visuomotor policies. Empirical results show that GLAM outperforms behavioral cloning baselines and previous latent-action methods, achieving an average task success rate improvement of 48% across five manipulation tasks, making it a significant advancement for practitioners dealing with diverse and unlabeled demonstration data.