VERIFIED TECHNICAL DOSSIERSources checked

Sebastian Raschka

Staff Research Engineer, Lightning AI | Author of Build a Large Language Model (From Scratch)

2 Verified ArtifactsSource Checked & Attributed

Back to Profile & Contact Inquire with Sebastian

Verified Proof of Work Artifacts

2 items cataloged

Each artifact below represents an authenticated research publication, production code repository, or technical architectural framework directly authored or co-created by Sebastian Raschka. Every entry undergoes editorial source verification.

RESEARCH Checked Sep 23, 2026

Practical LoRA & QLoRA Fine-Tuning Parameter Analysis & Benchmarks

Published empirical studies benchmarking the impact of rank (r), alpha scaling, target weight modules (Q, K, V vs all linear layers), and 4-bit NormalFloat quantization on model performance across Llama and Mistral.

Model & Execution Context:7B to 70B parameter models fine-tuned across consumer GPU hardware with LitGPT.

Scope & Limitations

Aggressive 4-bit quantization can induce subtle degradation on specialized domain syntax without rank compensation.

View Enriched Study & Permalinks Primary Source

IMPLEMENTATION Checked Sep 23, 2026

Build a Large Language Model (From Scratch): Complete PyTorch Implementation

Created the canonical open-source educational repository teaching engineers how to code, train, and fine-tune a complete GPT-style transformer from scratch in pure PyTorch without external high-level libraries.

Model & Execution Context:Modular Python/PyTorch codebase implementing multi-head self-attention, BPE tokenization, causal masking, and instruction alignment.

Scope & Limitations

Designed primarily for educational transparency; industrial scale multi-node training requires distributed FSDP/Megatron wrappers.

View Enriched Study & Permalinks Primary Source