Optimality of FSQ Tokens for Continuous Diffusion for Categorical Data with Application to Text-to-Speech

The paper presents a study on the optimality of FSQ tokenization for continuous diffusion models applied to categorical data, particularly in text-to-speech tasks. It demonstrates that the FSQ tokenization scheme optimally structures the latent space, yielding superior performance in token prediction as evidenced by theoretical analysis and numerical experiments. The findings indicate that text-to-speech models utilizing FSQ tokens outperform traditional LLM-based approaches while being smaller and faster, suggesting a promising alternative for practitioners in AI model development.