Substrate Asymmetry in User-Side Memory: A Diagnostic Framework

The paper presents a diagnostic framework for evaluating user-side memory in large language models (LLMs), revealing that memory can be decomposed into three distinct axes: behavioral consistency, factual presence, and factual absence. Through comparative analysis, it shows that a per-user gamma-LoRA adapter excels in maintaining behavioral style, while a retrieval-augmented generation (RAG) approach performs better in factual absence, highlighting a trade-off in performance based on the memory substrate used. The findings emphasize the alignment tax on parametric memory in heavily RLHF-tuned models like Llama-3.1-8B-Instruct, suggesting that practitioners need to consider these asymmetries when designing memory systems for LLMs.