RepSelect: Robust LLM Unlearning via Representation Selectivity

RepSelect (Representation Selectivity) is a new method for robust unlearning in large language models (LLMs) that addresses the limitations of existing approaches by isolating forget-set-specific representations. It achieves this by collapsing the top principal components of weight gradients before updates, which preserves general capabilities while significantly reducing the ability of fine-tuning to recover forgotten knowledge. Evaluations across models like Llama 3 and Qwen 3.5 demonstrate that RepSelect achieves a 4-50x greater reduction in post-relearning accuracy compared to existing baselines, marking a significant advancement in deep and robust LLM forgetting strategies.