Sebastian Raschka
Staff Research Engineer, Lightning AI | Author of Build a Large Language Model (From Scratch)
Staff Research Engineer at Lightning AI, author of the bestselling books 'Build a Large Language Model (From Scratch)' and 'Python Machine Learning', and creator of the Ahead of AI newsletter read by over 100,000 engineers. Former Assistant Professor of Statistics at the University of Wisconsin-Madison. Globally recognized for hands-on, first-principles code walkthroughs of LLM pretraining, LoRA/QLoRA fine-tuning, and PyTorch deep learning systems.
Areas of focus
Professional niches
Proof of Work
Practical LoRA & QLoRA Fine-Tuning Parameter Analysis & Benchmarks
Published empirical studies benchmarking the impact of rank (r), alpha scaling, target weight modules (Q, K, V vs all linear layers), and 4-bit NormalFloat quantization on model performance across Llama and Mistral.
Aggressive 4-bit quantization can induce subtle degradation on specialized domain syntax without rank compensation.
Context: 7B to 70B parameter models fine-tuned across consumer GPU hardware with LitGPT.
View missionBuild a Large Language Model (From Scratch): Complete PyTorch Implementation
Created the canonical open-source educational repository teaching engineers how to code, train, and fine-tune a complete GPT-style transformer from scratch in pure PyTorch without external high-level libraries.
Designed primarily for educational transparency; industrial scale multi-node training requires distributed FSDP/Megatron wrappers.
Context: Modular Python/PyTorch codebase implementing multi-head self-attention, BPE tokenization, causal masking, and instruction alignment.
View mission