Dr. Colin Raffel
University of Toronto / Vector Institute
Associate Professor of Computer Science at the University of Toronto, Faculty Member at Vector Institute, and former Research Scientist at Google Brain. Co-creator of the landmark T5 (Text-to-Text Transfer Transformer) model and pioneer of parameter-efficient fine-tuning (PEFT) and the open BigScience BLOOM initiative.
Areas of focus
Professional niches
Proof of Work
Parameter-Efficient Transfer Learning for NLP (Foundational PEFT Adapters)
Co-authored the landmark ICML 2019 paper introducing lightweight adapter modules injected into transformer layers, proving models could learn new tasks with less than 3% added parameters.
Serial adapter bottlenecks add minor inference latency compared to re-parameterization techniques like LoRA.
Context: Bottleneck projection adapters inserted after multi-head attention and feedforward layers.
View missionExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5)
First author of the seminal T5 paper establishing the unified text-to-text paradigm across all NLP tasks, introducing the C4 (Colossal Clean Crawled Corpus) dataset with over 25,000 academic citations.
Encoder-decoder architecture has higher KV-cache memory requirements during generation than pure autoregressive decoders.
Context: Encoder-decoder transformer architecture up to 11B parameters trained on C4 web corpus.
View mission