Evaluating Pluralism in LLMs through Latent Perspectives

The paper introduces a domain-agnostic multi-layered framework for the unsupervised extraction of perspectives in large language models (LLMs), aimed at addressing the pluralistic gap in LLM-generated text. Evaluated on a dataset of book reviews, the framework reveals that while certain models and prompting methods can capture a wide range of perspectives, they still struggle with representing rarer viewpoints, leading to a divergence from human-like distributions. This work is significant for practitioners as it highlights the limitations of current LLMs in generating diverse perspectives, informing future model training and evaluation strategies to enhance pluralistic alignment.