Structured Recurrent Mixers for Massively Parallelized Sequence Generation

The article introduces the Structured Recurrent Mixer (SRM), a novel architecture that enables the conversion between sequence parallel representation during training and recurrent representation during inference, optimizing both efficiency and throughput. Experimental results indicate that SRMs achieve 12x higher throughput and 170x greater concurrency compared to traditional Transformers, alongside a 30% increase in compute-constant GSM8k Pass@k performance. This architecture is particularly significant for practitioners as it enhances the handling of extended sequence lengths and information-rich inputs while maintaining training efficiency and scalability in batch processing.