Repeated post-training is not Self-improving: Diagnosing Scientific Amnesia in Continual DPO Pipelines

The paper introduces the concept of "scientific amnesia" in continual DPO training pipelines for LLMs, where models fail to accumulate reusable knowledge despite preserving learned behaviors. It presents a diagnostic suite for identifying this issue, a pipeline utilizing FSDP-sharded DPO checkpoints on Qwen2.5-7B-Instruct, and a benchmark based on 30 campaigns of HumanEval. The findings indicate that while many strategies degrade model performance, a conservative rule-based scheduling approach shows improvement, highlighting the need for tailored interventions based on specific training conditions and evaluation designs.