UniRank: Unified Rank Allocation for Low-Rank LLM Compression

The paper introduces UniRank, a novel method for low-rank allocation in the compression of large language models, addressing the inefficiencies of manual and learning-based approaches. It utilizes a sorting-and-truncation pipeline to score singular components based on local singular energy ratio and global functional importance, demonstrating a strong correlation between input-output cosine similarity and effective rank. Empirical results indicate that UniRank can achieve up to a 50% reduction in perplexity during one-shot compression without further fine-tuning, making it a significant advancement for practitioners aiming to optimize model performance while reducing computational overhead.