SICI: A Semantic-Pragmatic Complexity Index Reveals Regime Shifts in LLM Stance Detection

The article introduces the Stance Inference Complexity Index (SICI), a seven-dimensional measure designed to assess the semantic-pragmatic complexity of text pairs in stance detection tasks using prompt-based large language models (LLMs). SICI demonstrates superior predictive accuracy for LLM performance compared to traditional metrics, with a reliability score of $\alpha=0.771$, and reveals that LLM errors exhibit regime shifts based on complexity levels, influencing model behavior across various architectures, including GPT-3.5 and GPT-4o. This framework is significant for practitioners as it identifies the limitations of current prompting techniques and suggests that complexity reduction strategies may not effectively address high-complexity challenges in stance detection.