Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts

This study introduces a novel symmetric human-LLM evaluation framework based on Q methodology to assess value-structure alignment in large language models (LLMs). The researchers evaluated 12 LLMs across four model families using a 140-item moral statement set, revealing significant heterogeneity in model performance and sensitivity to generation stochasticity. This approach highlights the importance of structural evaluation in understanding moral reasoning capabilities of LLMs, suggesting that traditional benchmarks may overlook critical regional misalignments in value prioritization.