Guillaume Lample
Co-Founder & Chief Scientist
Co-founder & Chief Scientist at Mistral AI, and former research scientist at Meta AI. Led the creation of Mistral 7B, Mixtral 8x7B, and Mistral Large, and co-authored the original LLaMA foundation model, driving high-efficiency sparse mixture-of-experts into mainstream production.
Areas of focus
Professional niches
Proof of Work
Mixtral of Experts: Sparse Mixture of Experts with Open Weights
Co-authored the paper introducing Mixtral 8x7B, a decoder-only model where each layer is composed of 8 feedforward blocks with a router network that selects 2 experts per token, outperforming Llama 2 70B while maintaining the inference speed of a 12B model.
MoE models require substantial VRAM for weight storage despite sparse compute activations during generation.
Context: Sparse MoE, 47B total parameters, 13B active per token, 32k context with RoPE.
View missionLLaMA: Open and Efficient Foundation Language Models
Core co-author of the original Meta LLaMA 7B-65B release, proving that smaller models trained on significantly more tokens than Chinchilla recommendations surpass larger architectures and enable democratized research.
Base pre-trained checkpoints require safety and instruction fine-tuning to prevent conversational misalignment.
Context: SwiGLU activations, RMSNorm, Rotary Embeddings, 1.4T tokens.
View mission