VERIFIED TECHNICAL DOSSIERSources checked

Dr. Sasha Rush

Cornell Tech / Hugging Face

2 Verified ArtifactsSource Checked & Attributed

Back to Profile & Contact Inquire with Dr.

Verified Proof of Work Artifacts

2 items cataloged

Each artifact below represents an authenticated research publication, production code repository, or technical architectural framework directly authored or co-created by Dr. Sasha Rush. Every entry undergoes editorial source verification.

RESEARCH Checked Sep 21, 2026

Flash-Decoding: Accelerating Long-Context LLM Inference via Partitioned Attention

Co-developed Flash-Decoding, an inference parallelization algorithm that partitions key-value caches across thread blocks during token generation, delivering up to 8x speedup on 64k+ context sequences.

Model & Execution Context:CUDA kernels, split-KV cache reduction, integrated into vLLM, TensorRT-LLM, and PyTorch 2.2.

Scope & Limitations

Requires reduction step across partitions that introduces minor overhead for very short sequence lengths.

View Enriched Study & Permalinks Primary Source

EXPLANATION Checked Sep 21, 2026

The Annotated Transformer: A Line-by-Line PyTorch Implementation

Authored the universally referenced educational codebase providing an exact, executable line-by-line PyTorch implementation of 'Attention Is All You Need', bridging theoretical tensor math with production code.

Model & Execution Context:PyTorch tensor operations, multi-head attention, positional encoding, and beam search decoding.

Scope & Limitations

Presents vanilla quadratic multi-head attention without later FlashAttention or grouped-query optimizations.

View Enriched Study & Permalinks Primary Source