Neel Nanda
Mechanistic Interpretability Lead, Google DeepMind | Creator of TransformerLens
Lead of the Mechanistic Interpretability team at Google DeepMind and formerly Anthropic alignment researcher. Creator of TransformerLens, the de facto standard open-source research library for reverse-engineering the internal circuits, attention heads, and residual stream activations of transformer models. Internationally celebrated educator demystifying mechanistic interpretability through accessible tutorials and open challenges.
Areas of focus
Professional niches
Proof of Work
TransformerLens: Open Source Mechanistic Interpretability Framework
Created and maintains the premier open-source library allowing researchers to hook into, intervene upon, and cache intermediate activations across 50+ autoregressive transformer architectures with PyTorch integration.
Activation patching on models larger than 70B parameters requires substantial distributed multi-GPU memory.
Context: Hook point engine supporting GPT-2, Llama, Pythia, Mistral, and custom transformer variants.
View missionProgress Measures for Grokking via Mechanistic Interpretability
Reverse-engineered the exact circular Fourier representation circuits formed when transformers 'grok' modular addition long after overfitting, proving that continuous generalization progresses silently before discontinuous performance jumps.
Toy model arithmetic circuits do not fully capture superposition and polysemanticity present in production-scale language models.
Context: One-layer transformers trained on modular arithmetic tasks with weight decay.
View mission