How Transparent is DiffusionGemma?

The paper investigates the transparency of DiffusionGemma, a diffusion model, compared to the autoregressive Gemma 4 model. It finds that while DiffusionGemma initially exhibits poor variable transparency due to a high opaque serial depth (28.6X), this can be mitigated by mapping information through an interpretable token bottleneck, reducing the depth to 1.1X. The study also highlights challenges in achieving algorithmic transparency in diffusion models and presents novel phenomena observed during interpretability case studies, concluding that DiffusionGemma maintains similar monitorability to Gemma 4, which is crucial for practitioners focused on model interpretability and downstream task performance.