The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning

This work addresses the "Two-Hump" problem in mathematical reinforcement learning, particularly in the context of the Andrews-Curtis conjecture, by introducing novel data generation techniques and algorithmic enhancements, including supermoves and Transformer-based architectures. The authors release two comprehensive benchmark datasets: AC-19, containing 125,192 trivial instances, and AC-1M, with 1,136,154 hard instances, which are the first large-scale, publicly available datasets for this problem. These advancements are significant for practitioners as they enhance the training landscape for RL models by providing a richer set of problem instances, facilitating more effective learning in sparse reward environments.