Interactor: Agentic RL oriented Iterative Creation for Ad Description Generation in Sponsored Search

The paper introduces Interactor, a multi-turn iterative framework for generating ad descriptions in sponsored search, utilizing agentic reinforcement learning (RL). The model leverages a customized environment with generative reward models (GenRMs) to evaluate ad descriptions on multiple dimensions, including knowledge capacity and consistency with landing pages, iteratively refining outputs based on feedback. This approach demonstrates significant improvements over existing methods in generating informative ad content and has been deployed in a major search ads system, enhancing both ad revenue and user experience.