Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models

The article introduces LEAF-X, an explainable AI framework designed for transformer-based automatic speech recognition (ASR) models like Whisper. LEAF-X utilizes entropy-guided attention weighting and multi-layer attention rollout to enhance interpretability by producing sparse token-to-frame attributions, achieving a 32% improvement in faithfulness and 35-39% enhancement in locality and sparsity of explanations. This advancement is significant for practitioners as it provides more reliable insights into model behavior, facilitating transparency and auditability in ASR systems.