RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulation

The article introduces the Reference-Anchored Reward Model (RARM), a novel approach to reward modeling in reinforcement learning for robot manipulation. RARM leverages a lightweight visual comparator trained on general-purpose videos to generate dense, progress-aware rewards without requiring task-specific data or engineering. It demonstrates superior performance across nine simulated tasks and four real-world tasks, particularly excelling in long-horizon scenarios like cloth folding, thus addressing the challenges of reward design in complex manipulation tasks.