video-SALMONN-R$^3$: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding

The paper presents video-SALMONN-R$^3$, an end-to-end video large language model designed for efficient video understanding through a two-stage approach that includes re-watching segments at higher fidelity. This model leverages reinforcement learning to eliminate the need for costly chain-of-thought data annotations and incorporates a re-answer strategy to enhance response accuracy after re-watching, along with a re-ask mechanism to maintain query relevance. Experimental results indicate that video-SALMONN-R$^3$ outperforms previous models and benchmarks while reducing computational costs, making it a significant advancement for practitioners focused on video question answering.