CycliST: A Video Language Model Benchmark for Reasoning on Cyclical State Transitions

CycliST is a new benchmark dataset aimed at assessing Video Language Models (VLM) on their reasoning capabilities regarding cyclical state transitions in video sequences. It features a tiered evaluation system that increases complexity through variations in cyclic objects and scene conditions, revealing significant limitations in current VLMs' abilities to understand and exploit cyclical dynamics and temporal changes in visual attributes. This benchmark highlights the need for improvements in spatio-temporal cognition among VLMs, as no existing model consistently outperforms others across all tasks, emphasizing a critical area for future research and development in visual reasoning.