Barret Zoph
Member of Technical Staff, OpenAI | Co-Inventor of Neural Architecture Search (NAS) & ST-MoE
Verified Proof of Work Artifacts
2 items catalogedEach artifact below represents an authenticated research publication, production code repository, or technical architectural framework directly authored or co-created by Barret Zoph. Every entry undergoes editorial source verification.
ST-MoE: Designing Stable and Transferable Sparse Mixture-of-Experts Models
Authored the foundational engineering playbook for training multi-hundred-billion parameter Sparse Mixture-of-Experts (MoE) models, introducing the router z-loss to eliminate numerical instability and routing collapse.
Sparse routing creates communication-heavy all-to-all cross-accelerator collective operations that require high-bandwidth interconnects (NVLink/InfiniBand).
Neural Architecture Search with Reinforcement Learning
Introduced Neural Architecture Search (NAS), employing a recurrent neural network controller trained via reinforcement learning to generate model descriptions and automate neural network architecture design, winning seminal citations.
Original NAS required massive compute expenditures (thousands of GPU hours) before differentiable architecture search (DARTS) was invented.