OFMU: Optimization-Driven Framework for Machine Unlearning

The article presents OFMU, a penalty-based bi-level optimization framework designed for machine unlearning, which allows large language models to remove specific knowledge while maintaining performance on remaining data. OFMU employs a hierarchical structure with an inner maximization step that incorporates a similarity-aware penalty to mitigate conflicting gradients, and an outer minimization step to restore model utility. The framework demonstrates improved forgetting efficacy and retained utility compared to existing methods, with extensive experimental validation across various vision and language benchmarks, making it a significant advancement for practitioners needing effective unlearning capabilities in sensitive applications.