MemoryVAM: Integrating Memory into Video Action Model for Robot Manipulation

MemoryVAM introduces an episodic memory mechanism for video-world-model policies, enhancing long-horizon manipulation tasks by integrating a Recap-Cue module that compresses per-frame CLIP embeddings into memory tokens. This model employs a lightweight Cue Gate for task completion estimation and can be applied to various backbones, including UNet and Diffusion Transformer, with significant improvements in performance on the LIBERO-Mem benchmark, raising average success rates from 5% to 42.5%. For practitioners, these advancements in memory integration are crucial for developing more robust AI systems capable of handling complex, temporally extended tasks in real-world robot manipulation scenarios.