Repeated Shared Access Enables Grokking, but Edit Propagation Depends on a Fine-Grained Addressable Memory

The study investigates the role of repeated shared access in grokking and the effects of fine-grained addressable memory on factual edit propagation across four architectures: Dense, Loop, Dense+Mem, and LMC. Results indicate that while looped recomputation and memory rereading enable out-of-distribution grokking, effective edit propagation is contingent on the architecture's memory capabilities, with LMC showing a significant advantage in propagating edits (0.78-0.92) compared to non-memory architectures. This research highlights the importance of memory structures in enhancing both learning and editing capabilities in AI models, which is crucial for practitioners developing systems that require robust knowledge integration and manipulation.