LoSoNA: A Benchmark for Local Social Norm Adaptation in Group Conversations

LoSoNA is a newly introduced benchmark designed to assess local social norm adaptation in multi-party conversations for LLMs. The benchmark evaluates eight models, including Gemini 3.1 Pro and Claude Fable 5, under varying prompting conditions, revealing that explicit norm-aware prompting significantly improves performance, with Gemini 3.1 Pro achieving 84.2% accuracy. This benchmark is crucial for practitioners as it highlights the importance of social context in LLM interactions, guiding the development of more socially aware AI systems.