Andrej Karpathy
Founder, Eureka Labs | Former Director of AI at Tesla & OpenAI Founding Member
Andrej Karpathy is an AI researcher and educator who founded Eureka Labs, an AI-native education company. Previously, Andrej served as the Director of AI at Tesla, leading the Autopilot computer vision and neural network architecture teams, and was a founding research scientist at OpenAI. Widely revered as one of the world's premier deep learning educators, he created nanoGPT and the landmark 'Neural Networks: Zero to Hero' lecture series.
Areas of focus
Professional niches
Proof of Work
llm.c: Large Language Model Training in Pure C/CUDA Without Dependencies
A zero-dependency, pure C and CUDA implementation of GPT-2 and modern transformer pre-training. Eliminates the 245MB+ PyTorch/Python runtime overhead by directly implementing forward passes, backward passes, AdamW optimizers, and fused CUDA kernels in a single clean compilation unit, achieving near-parity with PyTorch cuDNN speed.
Lower level of architectural expressiveness compared to dynamic Python autograd graphs; adding novel attention variations requires manual mathematical derivation and custom CUDA kernel writing.
Context: Pure C99, CUDA 12, FP32 and BF16 mixed-precision kernels, fused attention, multi-GPU MPI execution.
View missionmicrograd: Minimal Scalar-Valued Autograd Engine with Dynamic DAG Visualization
A foundational, pedagogical autograd engine implementing reverse-mode autodiff over dynamically constructed Directed Acyclic Graphs (DAGs) in approximately 100 lines of pure Python, alongside a multi-layer perceptron (MLP) library that replicates PyTorch's core tensor mechanics from first principles.
Operates on individual scalar values rather than vectorized multidimensional tensors, making it unsuitable for training production models beyond small educational datasets.
Context: Pure Python 3.10+, scalar-valued autodiff DAG, Graphviz topological computation graph visualization.
View missionAutoResearch: Autonomous AI Research Agent Conducting Iterative ML Experiments
An autonomous multi-step research agent framework designed to conduct end-to-end machine learning experimentation without human intervention. The agent autonomously formulates algorithmic hypotheses, modifies PyTorch model architectures via AST rewriting, schedules multi-GPU training jobs, parses validation loss curves, and generates structured empirical research reports and pull requests.
Requires sandboxed execution environments to prevent arbitrary code execution risks during dynamic model definition; experiments can diverge if loss curves suffer from exploding gradients without bounded learning rate search spaces.
Context: PyTorch 2.4, LLM function calling, Automated Evaluation Loop, Python AST manipulation, TensorBoard metric parsing.
View missionNeural Networks: Zero to Hero Video Lecture Curriculum & Repository
A masterclass educational curriculum and open-source codebase tracing neural networks from basic micrograd autograd engines through multi-layer perceptrons, WaveNet, and full Transformer language models. Authored by Andrej Karpathy to provide deeply intuitive, first-principles deep learning literacy.
Pedagogical and reference implementation. Optimized for code clarity, educational inspection, and conceptual clarity rather than maximum distributed hardware throughput.
Context: Pure PyTorch and custom Python micrograd engine; runnable on consumer GPUs or Google Colab.
View missionnanoGPT: Minimal, Readable, and Fast PyTorch Framework for Training Medium GPTs
A pure, clean, and hackable 300-line PyTorch implementation for training, fine-tuning, and evaluating transformer language models with support for FlashAttention and torch.compile.
Engineered for educational transparency and small-to-medium parameter models; multi-node distributed trillion-token training requires complex pipeline parallel frameworks like Megatron.
Context: PyTorch 2.0, CUDA, Python, FlashAttention, and OpenWebText.
View mission