Manifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Models

The paper introduces Bayesian Manifold Curriculum (BMC), a novel framework for reinforcement learning that organizes problems into a hierarchical task tree and employs Bayesian learning to optimize problem sampling in large language models (LLMs). By framing problem selection as a manifold-structured bandit problem, BMC accounts for the relationships within the model's latent representation space, leading to improved learning efficiency by balancing productivity, diversity, and utility. This approach emphasizes the necessity of structure-aware sampling strategies over traditional difficulty-based methods, offering insights for practitioners aiming to enhance LLM training processes.