Operads for compositional reasoning in LLMs

The paper introduces a mathematical framework using operads to formalize question decomposition in LLMs, addressing the lack of rigorous foundations in this approach. It defines the questions operad \( Q \) to represent question templates and their compositions, and proposes a novel metric called operadic consistency, which correlates with the accuracy of twelve LLMs across multi-hop QA datasets, outperforming traditional self-consistency methods. This framework offers practitioners new tools for enhancing the reliability of multi-step reasoning in AI applications.