Arthur Mensch
AI Founder
Verified Proof of Work Artifacts
2 items catalogedEach artifact below represents an authenticated research publication, production code repository, or technical architectural framework directly authored or co-created by Arthur Mensch. Every entry undergoes editorial source verification.
Mistral 7B: Grouped-Query and Sliding Window Attention Foundation Model
Designed and published Mistral 7B, an open-weights autoregressive transformer utilizing Grouped-query Attention (GQA) and Sliding Window Attention (SWA) to achieve faster inference and handle 8k context sequences while beating Llama 2 13B across all standard benchmarks.
Requires high cache memory bandwidth on commodity GPUs; sliding window attention drops exact attention history beyond context horizon without caching adaptations.
Mixtral of Experts: Sparse Mixture-of-Experts Open Weight Model
Published the architectural blueprint for Mixtral 8x7B, a sparse mixture-of-experts model where a router chooses 2 out of 8 feedforward experts per token, activating only 13B parameters out of 47B during forward pass and outperforming Llama 2 70B with 6x faster throughput.
High RAM/VRAM memory footprint for hosting all 47B parameters despite activating only 13B during computation; requires specialized multi-GPU tensor or pipeline parallelism.