Whisper-CD: Accurate Long-Form Speech Recognition using Multi-Negative Contrastive Decoding

Whisper-CD is a novel contrastive decoding framework designed to enhance long-form speech recognition in large encoder-decoder models like Whisper, addressing issues such as hallucinations and repetition loops. It employs a training-free approach that contrasts clean-audio logits with negative logits derived from Gaussian noise, silence signals, and audio temporal shifts, achieving up to a 24.3 percentage point reduction in word error rate (WER) on the CORAAL benchmark and 48% faster token generation compared to traditional beam search. This drop-in solution allows practitioners to improve existing Whisper systems without the need for retraining, making it a practical enhancement for real-world applications.