The Culture Funnel: You Can't Align What isn't in the Data

The authors present a new culturally tagged dataset comprising 5.6 million samples, aimed at addressing the deficiencies in cultural knowledge within current LLM training pipelines. They highlight that cultural signals diminish during post-training due to the predominance of geographically concentrated, task-specialized data, and propose a multidimensional tagging framework to enhance cultural representation in AI models. This dataset release is significant for practitioners as it provides a resource to improve cultural alignment in LLMs, potentially leading to better performance on cultural benchmarks.