Can In-Context Learning Support Intrinsic Curiosity?

The paper investigates the potential of in-context learning (ICL) in supporting intrinsic curiosity for automated data selection in machine learning. It reveals that while traditional methods for incentivizing exploration through learning progress are computationally impractical, ICL can serve as immediate, update-free world models. The authors demonstrate that ICL-derived rewards can effectively guide exploration policies in non-temporal settings, leading to optimal data collection strategies, which is significant for practitioners aiming to enhance exploration efficiency in AI systems.