LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing

LOKI is a new method for lifelong knowledge editing in language models that utilizes dynamic layer selection via the Hilbert-Schmidt Independence Criterion and projects gradient updates onto the null-space of model weights. This approach circumvents the need for access to previous knowledge and extensive pre-processing, addressing issues of catastrophic forgetting while enhancing flexibility. In experiments, LOKI demonstrated up to a 14% improvement in average accuracy over existing methods, making it a significant advancement for practitioners focused on efficiently updating AI models.