MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards

MemBuilder is a newly introduced reinforcement learning framework designed to enhance long-term memory construction in LLMs by utilizing attributed dense rewards. It tackles the issues of sparse trajectory-level rewards through synthetic session-level question generation and implements multi-dimensional memory attribution via contribution-aware gradient weighting, leading to improved training efficiency. Experimental results demonstrate that a 4B-parameter model trained with MemBuilder outperforms existing closed-source models on long-term dialogue benchmarks, making it a significant advancement for practitioners focused on developing more consistent and context-aware dialogue systems.