Tim Dettmers
Staff Research Scientist, Ai2 | Creator of bitsandbytes & QLoRA Quantization
Tim Dettmers is a Staff Research Scientist at the Allen Institute for AI (Ai2) and former Postdoc at the University of Washington. Tim is the creator of bitsandbytes and co-author of QLoRA, breakthrough open-source technologies that democratized high-performance 4-bit and 8-bit quantization and fine-tuning on consumer GPUs.
Areas of focus
Professional niches
Proof of Work
8-Bit Optimizers via Block-wise Dynamic Quantization with Low-Precision States
A breakthrough algorithmic paper by Tim Dettmers introducing 8-bit Adam and low-precision optimizer states, reducing memory consumption of deep learning training states by 75% without compromising model convergence.
Custom CUDA block-level quantization; optimizer speedups are maximized on modern Tensor Core architectures.
Context: Benchmarked on language modeling, machine translation, and ImageNet classification with PyTorch.
View missionbitsandbytes: 8-Bit and 4-Bit CUDA Quantization Primitives
The open-source library behind 8-bit matrix multiplication and 4-bit NormalFloat quantization, enabling consumer GPUs to run and train 70B+ parameter foundation models with negligible accuracy loss.
Tightly coupled with NVIDIA CUDA architectures; requires compilation of custom C++/CUDA kernels for optimal memory coalescing.
Context: CUDA, C++, Python, Hugging Face transformers, PEFT, and PyTorch.
View mission