SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression

SVD-Surgeon is a novel training-free method for compressing large language models (LLMs) using singular value decomposition (SVD), enhancing the Optimal Brain Surgeon (OBS) framework. It optimally updates retained singular values to mitigate the loss from truncation and provides a saliency measure for pruning decisions. When applied to the SVD-LLM method, SVD-Surgeon demonstrates improved perplexity-compression trade-offs on the OPT family and LLaMA 2-7B models, making it a valuable tool for practitioners looking to optimize LLM deployment without retraining.