Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

The paper introduces UDS (Utility-Diversity Sampling), a novel framework for online batch selection in supervised fine-tuning (SFT) of large language models (LLMs). UDS enhances computational efficiency by combining utility and diversity metrics using the nuclear norm of the logits matrix and low-dimensional embeddings, eliminating the need for external resources and reducing unnecessary backpropagation. Experimental results indicate that UDS outperforms existing online batch selection methods across various benchmarks, offering significant training time savings compared to traditional full-dataset fine-tuning.