Balancing Performance and Diversity in GRPO Autoregressive Text-to-Image Post-Training

The paper presents a novel approach to autoregressive text-to-image generation by employing a GRPO-style online reinforcement learning framework that dynamically addresses reference-policy divergence using a unified f-divergence framework. Key findings indicate that using Jensen-Shannon (JS) divergence for policy optimization enhances both performance and diversity in generated outputs, outperforming existing methods in experiments conducted on LlamaGen and Janus-7B. This work is significant for practitioners as it provides a theoretically grounded method to improve alignment with human preferences while maintaining generation diversity, which is crucial for developing more robust T2I models.