
Mistral
Overview
Mistral was introduced in the this blogpost by Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.
The introduction of the blog post says:
Mistral AI team is proud to release Mistral 7B, the most powerful language model for its size to date.
Mistral-7B is the first large language model (LLM) released by mistral.ai.
Architectural details
Mistral-7B is a decoder-only Transformer with the following architectural choices: