PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation

The article introduces PRIDE (Privileged Information-enhanced Distillation), a method designed to improve empathetic dialogue generation by effectively transferring knowledge from larger teacher models to smaller student models without requiring additional inputs during inference. Key technical components include an empathy-reasoning prompt, a multi-source attention mechanism, and a dual-alignment loss that utilizes reversed Kullback-Leibler divergence and maximum mean discrepancy. Experiments show that PRIDE achieves competitive performance on both multi-modal and text-only datasets, matching or exceeding the capabilities of larger models, which is significant for practitioners seeking efficient deployment of empathetic dialogue systems in resource-constrained environments.