CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters

The article introduces CuMA (Cultural Mixture of Adapters), a framework designed to align large language models (LLMs) with diverse cultural values by addressing the issue of Mean Collapse caused by Cultural Sparsity. CuMA utilizes demographic-aware routing and a Latent Cultural Topology to separate conflicting gradients into specialized expert subspaces, achieving state-of-the-art performance on benchmarks such as WorldValuesBench and PRISM. This approach is significant for practitioners as it preserves cultural diversity in LLM outputs, enabling models to better serve a global audience with varying cultural contexts.