Reinforcement Learning Foundation Models Should Already Be A Thing

The article proposes the development of reinforcement learning (RL) foundation models, highlighting the potential for sampling synthetic Markov Decision Processes (MDPs) akin to synthetic tabular datasets. It introduces a Graph Attention Network trained on synthetic MDPs, demonstrating that it outperforms traditional methods like UCB-VI and tabular Q-learning in online scenarios and competes effectively with VI-LCB in offline contexts, thus suggesting a new direction for RL model design that leverages attention-based architectures. This advancement could significantly enhance the efficiency and performance of RL applications by integrating foundational principles similar to those in language and vision models.