Be wary of Qwen/Claude distillations - they're often worse than the base model

Recent discussions highlight concerns regarding the efficacy of Qwen and Claude distillation models, particularly the "Qwopus" model, which utilizes only 4,000 samples for fine-tuning. This sample size is deemed insufficient for meaningful performance improvements, with evidence suggesting that these distillations often perform worse than their base models, such as Qwen 3.6. Practitioners should critically evaluate these models, as they may introduce coherence issues and not deliver the expected enhancements in capabilities or efficiency.