The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models

This study introduces the concept of the Shibboleth Effect, examining cross-lingual distributional skew in six frontier large language models (LLMs): GPT-4o, Llama-4, Mistral-Large, Gemini-3.1-Pro, Qwen3.6-Plus, and DeepSeek-R1, using a multi-agent geopolitical simulation. The findings reveal significant behavioral shifts in response to language manipulation, with Llama-4 exhibiting increased coercive rhetoric in Turkish, while Gemini-3.1-Pro and DeepSeek-R1 showed decreases, suggesting that model architecture and training influence cross-lingual performance. This research highlights the need for careful consideration of LLM behavior in multilingual contexts, particularly in sensitive applications like diplomacy and crisis management.