DFlash Speculative Decoding Drafts Whole Token Blocks in Parallel for Up to 15x Higher Throughput on NVIDIA Blackwell

UC San Diego's DFlash introduces a block diffusion model for speculative decoding, allowing for the drafting of whole token blocks in a single forward pass with key-value (KV) injection for conditioning. The model achieves a reported 6.08x lossless speedup on the Qwen3-8B model and up to 15x throughput on NVIDIA's Blackwell architecture, while supporting frameworks like SGLang, vLLM, and TensorRT-LLM. This advancement is significant for practitioners as it enhances decoding efficiency and throughput, which are critical for real-time applications in AI.