ARROW: Augmented Replay for RObust World models

ARROW (Augmented Replay for RObust World models) is a new model-based continual reinforcement learning algorithm that enhances DreamerV3 with a memory-efficient, distribution-matching replay buffer. It features two complementary buffers: a short-term buffer for recent experiences and a long-term buffer for task diversity through intelligent sampling. In evaluations on Atari and Procgen CoinRun tasks, ARROW significantly reduces catastrophic forgetting compared to existing model-free and model-based baselines while maintaining forward transfer, underscoring the effectiveness of bio-inspired approaches in continual RL.