LLM-Assisted Stance Detection in Scientific Discourse: A Test Case in Bayesian Cognitive Science

The study introduces a framework for stance detection in scientific discourse, specifically examining the interpretation of Bayesian models in cognitive science. It employs a diagnostic-gated prompt-optimization method across three LLMs (GPT-5.1, Claude Sonnet 4.6, Gemini 3 Pro Preview), achieving a combined reliability score of 0.76 and substantial agreement in quote-level analysis (ICC = 0.80). This work is significant for practitioners as it demonstrates the potential of LLMs to assist in nuanced qualitative coding, providing a structured approach that may be adapted for other complex interpretive tasks in social sciences.