The Anatomy of the CTC Oracle Gap: Acoustic Exhaustion and Linguistic Recovery

The study examines the limitations of Connectionist Temporal Classification (CTC) in N-best hypothesis selection, revealing that the discriminative capacity of CTC-internal representations is saturated, as evidenced by a significant drop in correlation between hypothesis scores and word error rate (WER) with increased beam size. The introduction of external linguistic information via Minimum Bayes Risk (MBR) decoding using a RoBERTa pseudo-log-likelihood leads to a notable WER reduction to 5.42% on the LibriSpeech test set, outperforming greedy decoding by 0.535 percentage points. This research highlights the importance of integrating linguistic context in acoustic models to improve performance, especially in diverse environments and noise conditions.