ScalingAttention: Discovering Intrinsic Sparse Attention Topology for Video Diffusion Transformers

The paper introduces ScalingAttention, a framework designed to optimize Diffusion Transformers (DiTs) by addressing the computational limitations of 3D full attention, which has a quadratic complexity. It features two key innovations: WEST, which generates a robust block-sparse prior mask without runtime overhead, and FAST, which adjusts head-wise sparsity based on fidelity needs. Experimental results on the Wan2.1 dataset demonstrate up to 1.90X speedup while maintaining superior video quality, marking significant progress in efficient video generation for practitioners.