Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech

The paper introduces a code-mixing guided preference-learning framework to enhance code-switching Automatic Speech Recognition (ASR) by improving synthetic speech generation. It employs the Code Mixing Index (CMI) to ensure language-boundary consistency, resulting in significant reductions in Mixed Error Rate (MER) when fine-tuning the Whisper Large model on the SEAME Mandarin-English corpus. This approach addresses the challenge of limited high-quality code-switching data, providing a valuable method for practitioners looking to improve ASR performance in multilingual contexts.