Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo

The paper introduces MeMo, a framework utilizing multi-layer correlation matrix memories (CMMs) to facilitate version-aware operations in language models, allowing for efficient knowledge updates without full retraining. It proposes a version-aware operation layer that includes high-level functions such as replace, obsolete, and rollback, which are implemented as primitive calls over sequences and tokens. This architecture aims to enhance the adaptability of language models by enabling structured edits and maintaining historical data, thereby improving the efficiency and effectiveness of knowledge management in AI systems.