Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do

This paper investigates the effectiveness of Chain-of-Thought (CoT) reasoning in multimodal tasks, evaluating 12 tasks with 14 non-reasoning and 8 reasoning models. Key findings indicate that CoT can enhance performance in reasoning tasks but may hinder perception tasks like visual grounding, and that existing multimodal reasoning models show only marginal improvements due to a focus on mathematical reasoning. The study highlights a critical limitation in visual reasoning, where models struggle to maintain visual introspection during the reasoning process, suggesting a need for more balanced approaches in multimodal AI development.