STORM: Stepwise Token Optimization with Reward-Guided Beam Search

STORM (Stepwise Token Optimization with Reward-Guided Beam Search) is a new self-supervised framework designed for lexical query expansion, which optimizes rewriter performance by scoring candidate expansions against a BM25 index. It enables 0.6B-8B parameter models to match or exceed the effectiveness of larger LLM rewriters while maintaining the speed of traditional BM25 retrieval, and it demonstrates zero-shot transfer across 18 languages, outperforming dedicated multilingual dense retrievers. This approach offers practitioners a more efficient and infrastructure-light alternative for enhancing retrieval performance in AI systems.