0 citations
Mistral 7B
arXiv (Cornell University)2023
Citations Over Time
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas J. Wang, Timothée Lacroix, William El Sayed
Abstract
We introduce Mistral 7B v0.1, a 7-billion-parameter language model engineered for superior performance and efficiency. Mistral 7B outperforms Llama 2 13B across all evaluated benchmarks, and Llama 1 34B in reasoning, mathematics, and code generation. Our model leverages grouped-query attention (GQA) for faster inference, coupled with sliding window attention (SWA) to effectively handle sequences of arbitrary length with a reduced inference cost. We also provide a model fine-tuned to follow instructions, Mistral 7B -- Instruct, that surpasses the Llama 2 13B -- Chat model both on human and automated benchmarks. Our models are released under the Apache 2.0 license.
Related Papers
- → Prevalence and Evolution of License Violations in npm and RubyGems Dependency Networks(2022)12 cited
- → Licence Profile: Apache License, Version 2.0(2010)6 cited
- → Research and Application of the Floating License Management Strategy(2009)4 cited
- → A Window Inference Tool for Refinement(1992)23 cited
- → Sliding-Window Belief Propagation with Unequal Window Size for Nonstationary Heterogeneous Source(2020)