Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale

The article presents a "versioned late materialization" paradigm for training Deep Learning Recommendation Models (DLRMs) with ultra-long User Interaction History (UIH). This approach reduces storage and I/O overhead by utilizing a normalized, immutable storage layer and reconstructing sequences on-the-fly during training, which enhances efficiency in multi-tenant environments. Implemented in production DLRMs like HSTU and ULTRA-HSTU, this method significantly lowers resource usage while allowing for aggressive scaling of sequence lengths, ultimately improving model quality.