Tri Dao
Chief Scientist, Together AI & Assistant Professor, CMU | Inventor of FlashAttention
Verified Proof of Work Artifacts
2 items catalogedEach artifact below represents an authenticated research publication, production code repository, or technical architectural framework directly authored or co-created by Tri Dao. Every entry undergoes editorial source verification.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
A foundational architecture co-authored by Tri Dao and Albert Gu introducing selective state space models (SSMs). Mamba achieves linear scaling in sequence length with 5x higher inference throughput than standard Transformers while matching performance across million-token language contexts.
Requires specialized hardware-aware CUDA kernels for hardware utilization; memory benefits depend on hardware SRAM cache hierarchy.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Seminal hardware-aware algorithm optimizing GPU SRAM and HBM memory reads/writes to compute exact attention in sub-quadratic memory, delivering 2-4x speedups across major foundation models.
Requires GPU compute capability >= 8.0 (NVIDIA Ampere, Ada, or Hopper) and compiled CUDA/Triton kernels.