Timothée Lacroix
Co-Founder & Chief Technology Officer
Co-founder & CTO at Mistral AI and former research engineer at Meta AI. Architect of the distributed training clusters and high-performance inference pipelines powering Mistral's open and commercial models, specializing in sharded tensor parallelism and FP8 cluster optimization.
Areas of focus
Professional niches
Proof of Work
mistral-src: Reference Implementation and MoE Inference Engine
Built the official open-source reference codebase for Mistral 7B and Mixtral 8x7B, implementing sliding window attention (SWA), dynamic chunking, and Megatron-style tensor parallel execution for multi-GPU serving.
Sliding window attention truncates attention cache for earlier tokens outside the local window unless global tokens are designated.
Context: Sliding window attention (W=4096), grouped-query attention (GQA), PyTorch.
View missionLarge-Scale Distributed Pre-Training and Sharded Tensor Parallelism
Architected the cluster infrastructure utilizing Megatron-LM and custom NCCL collective communication topologies that pre-trained the Mixtral family across thousands of H100 GPUs with near-linear scaling efficiency.
Inter-node all-to-all communications in large MoE routing can bottleneck throughput on non-InfiniBand cloud interconnects.
Context: H100 NVLink clusters, InfiniBand HDR/NDR fabric, FP8 mixed precision.
View mission