Document Optimization for Black-Box Retrieval via Reinforcement Learning

The paper introduces a novel approach to document optimization for black-box retrieval systems, framing document expansion as a document optimization problem fine-tuned using reinforcement learning with GRPO, leveraging retrieval rank improvements as rewards. This method, applicable to various retriever types, shows significant performance gains, with the OpenAI text-embedding-3-small model improving nDCG5 scores on code retrieval from 58.7 to 66.8 and on visual document retrieval from 53.3 to 57.6, even outperforming the larger text-embedding-3-large model. The findings suggest that learned document transformations can enhance retrieval efficiency, particularly for smaller models, and combining this approach with retriever fine-tuning yields the best results, as demonstrated by improvements in Jina-ColBERT-V2.