Andrej Karpathy
Founder, Eureka Labs | Former Director of AI at Tesla & OpenAI Founding Member
Verified Proof of Work Artifacts
5 items catalogedEach artifact below represents an authenticated research publication, production code repository, or technical architectural framework directly authored or co-created by Andrej Karpathy. Every entry undergoes editorial source verification.
llm.c: Large Language Model Training in Pure C/CUDA Without Dependencies
A zero-dependency, pure C and CUDA implementation of GPT-2 and modern transformer pre-training. Eliminates the 245MB+ PyTorch/Python runtime overhead by directly implementing forward passes, backward passes, AdamW optimizers, and fused CUDA kernels in a single clean compilation unit, achieving near-parity with PyTorch cuDNN speed.
Lower level of architectural expressiveness compared to dynamic Python autograd graphs; adding novel attention variations requires manual mathematical derivation and custom CUDA kernel writing.
micrograd: Minimal Scalar-Valued Autograd Engine with Dynamic DAG Visualization
A foundational, pedagogical autograd engine implementing reverse-mode autodiff over dynamically constructed Directed Acyclic Graphs (DAGs) in approximately 100 lines of pure Python, alongside a multi-layer perceptron (MLP) library that replicates PyTorch's core tensor mechanics from first principles.
Operates on individual scalar values rather than vectorized multidimensional tensors, making it unsuitable for training production models beyond small educational datasets.
AutoResearch: Autonomous AI Research Agent Conducting Iterative ML Experiments
An autonomous multi-step research agent framework designed to conduct end-to-end machine learning experimentation without human intervention. The agent autonomously formulates algorithmic hypotheses, modifies PyTorch model architectures via AST rewriting, schedules multi-GPU training jobs, parses validation loss curves, and generates structured empirical research reports and pull requests.
Requires sandboxed execution environments to prevent arbitrary code execution risks during dynamic model definition; experiments can diverge if loss curves suffer from exploding gradients without bounded learning rate search spaces.
Neural Networks: Zero to Hero Video Lecture Curriculum & Repository
A masterclass educational curriculum and open-source codebase tracing neural networks from basic micrograd autograd engines through multi-layer perceptrons, WaveNet, and full Transformer language models. Authored by Andrej Karpathy to provide deeply intuitive, first-principles deep learning literacy.
Pedagogical and reference implementation. Optimized for code clarity, educational inspection, and conceptual clarity rather than maximum distributed hardware throughput.
nanoGPT: Minimal, Readable, and Fast PyTorch Framework for Training Medium GPTs
A pure, clean, and hackable 300-line PyTorch implementation for training, fine-tuning, and evaluating transformer language models with support for FlashAttention and torch.compile.
Engineered for educational transparency and small-to-medium parameter models; multi-node distributed trillion-token training requires complex pipeline parallel frameworks like Megatron.