Welcome Mixtral - a SOTA Mixture of Experts on Hugging Face

Mixtral, a state-of-the-art Mixture of Experts model, has been released on Hugging Face. It leverages a unique architecture with 12 experts and a total of 1.5 billion parameters, achieving superior performance on the GLUE benchmark with a score of 90.5. This model is significant for practitioners as it optimizes resource utilization while maintaining high accuracy, enabling efficient deployment in applications requiring large-scale language understanding.