Understanding the Behaviors of Environment-aware Information Retrieval

This study presents a systematic analysis of how large language models (LLMs) can adapt their query formulation strategies for different retrievers using reinforcement learning (RL). The findings indicate that retrievers have distinct optimal query styles, and the performance of retrieval-augmented generation (RAG) systems can be improved by incorporating retriever-specific human guidance and scaling model size. A novel branching-based rollout technique is introduced to enhance training stability over multi-retrieval-step trajectories, providing valuable insights for developing more effective retriever-aware RAG systems.