Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

The article introduces two new diversity evaluation metrics for generative AI models: Conditional-Vendi and Conditional-RKE, which are designed to assess prompt-induced variability in outputs. These metrics leverage conditional entropy from positive semidefinite matrices, with Conditional-RKE achieving an $O(1/\sqrt{n})$ convergence rate and Conditional-Vendi utilizing a truncated-spectrum approximation for scalability. The methods demonstrate effectiveness across various tasks, including text-to-image generation and image captioning, providing practitioners with improved tools for evaluating diversity in prompt-guided generation.