micrograd: Minimal Scalar-Valued Autograd Engine with Dynamic DAG Visualization
micrograd is Andrej Karpathy's seminal scalar autograd engine that strips deep learning backpropagation down to its absolute mathematical essence. While modern frameworks like PyTorch and JAX operate over high-dimensional multidimensional tensors, micrograd implements reverse-mode automatic differentiation over individual scalar values in just ~100 lines of pure Python.
At the heart of micrograd is the Value class, which wraps a single floating-point number, retains references to its parent operands in a Directed Acyclic Graph (DAG), and tracks local partial derivatives through closures. When .backward() is invoked, micrograd topologically sorts the execution graph and iteratively applies the multivariate chain rule from the final loss node back to the input parameters.
On top of this scalar engine, Karpathy constructs a functional Multi-Layer Perceptron (MLP) containing Neurons and Layers. Despite having zero dependencies and running entirely on CPU, micrograd is capable of training a binary classifier on a 2D dataset to 100% accuracy using simple gradient descent.
micrograd has achieved mythic status in the artificial intelligence education community, serving as the required foundational exercise for understanding how neural network gradients are computed before students encounter tensor broadcasting and CUDA memory layouts.
Implements reverse-mode automatic differentiation over scalar values in ~100 lines of standard Python.
Builds the computational graph on-the-fly during forward passes and traverses it in topological order during backprop.
Requires no external libraries (no numpy, no PyTorch); runs natively in pure Python standard library.
Includes dynamic graph visualization tools rendering complete computation trees and accumulated gradients.
Value Class & Scalar Wrapper
Encapsulates scalar float data and accumulated gradient, overloading standard Python arithmetic operators (+, *, **, tanh, relu).
Dynamic Computation DAG
Stores child node references and backward lambda closures during each forward mathematical operation.
Topological Sort & Backprop
Performs depth-first search to build a topological ordering of nodes, executing chain-rule backward passes in reverse dependency order.
Neural Network Abstraction
Constructs Neuron, Layer, and MLP classes with learnable weights and biases, optimized via stochastic gradient descent.
Pure Python 3.10+, scalar-valued autodiff DAG, Graphviz topological computation graph visualization.
Operates on individual scalar values rather than vectorized multidimensional tensors, making it unsuitable for training production models beyond small educational datasets.
How does micrograd differ from PyTorch autograd?
micrograd operates exclusively on scalar numbers (single floats), whereas PyTorch autograd operates on multidimensional tensors (n-dimensional arrays). The mathematical chain-rule logic is identical, but micrograd avoids tensor broadcasting, stride indexing, and GPU kernel management, making the core backprop algorithm 100% visible and understandable.
Why is topological sorting necessary during the backward pass?
In a computation graph, a node may feed into multiple downstream operations. If gradients were backpropagated without topological sorting, a node's gradient might be computed before all downstream contributions had accumulated. Topological sorting guarantees that every child node has fully received its gradients before its own parent backward closures are executed.
Can micrograd be used for production machine learning?
No. micrograd is intentionally designed for educational pedagogy and algorithmic clarity. Because it operates on scalars in pure Python without vectorized SIMD or GPU acceleration, training even small networks is thousands of times slower than PyTorch.
This proof of work artifact was source-checked on Sep 20, 2026 by the AI Experts Directory editorial team. Our source review confirms that public code repositories, research papers, and technical artifacts directly corroborate Andrej Karpathy's active contributions. For full verification criteria, read our editorial methodology.
Inspect original artifact sources
Review raw code repositories, benchmark datasets, and technical citations directly on github.com.