Learning Process Rewards via Success Visitation Matching for Efficient RL

The paper introduces a novel method for transforming sparse rewards in reinforcement learning (RL) into dense process rewards using a discriminator to differentiate between successful and unsuccessful episodes. This technique incentivizes the policy to match the state-action visitations of successful episodes, facilitating faster training without altering the optimal policy. The approach significantly improves finetuning performance in robotic control tasks, demonstrating its practical relevance for practitioners aiming to enhance RL efficiency in sparse reward scenarios.