Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning

The paper evaluates the reasoning capabilities of 25 large language models (LLMs) and human participants in common-sense scenarios, revealing that both exhibit similar error patterns indicative of pattern matching rather than true reasoning. The study identifies specific attention heads in LLMs that contribute to these reasoning errors, suggesting that both humans and LLMs rely on similar mechanisms for everyday reasoning. This insight is crucial for practitioners as it challenges the assumption of abstract reasoning in LLMs and highlights the importance of understanding the underlying mechanisms that govern model behavior in practical applications.