When Does a Video-Language Model Stop Watching? Reward Strength Controls the Formation and Reversal of Visual Shortcuts in Multimodal RLVR

The paper presents findings on the dynamics of visual shortcuts in large vision-language models (LVLMs) using reinforcement learning with verifiable rewards (RLVR). It introduces a grounding penalty parameter, lambda, which influences the formation and reversal of these shortcuts, revealing that shortcut reliance emerges abruptly and can be modulated by adjusting lambda. This research is significant for practitioners as it provides insights into the timing and strength of regularization strategies needed to maintain effective multimodal learning, thereby enhancing model robustness against perceptual biases.