AI research and projects
The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.
A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗
Dense Passage Retrieval for Open-Domain Question Answering (DPR)
Dr. Danqi Chen · Princeton UniversityPublished the landmark EMNLP 2020 paper demonstrating that dual-encoder BERT architectures trained on contrastive negative passages dramatically outperform classic BM25 and TF-IDF sparse retrieval in question answering.
Pure dense retrieval can struggle on out-of-domain lexical keywords and acronyms compared to hybrid dense-sparse search.
Context: Dual BERT encoders, dot-product similarity search over FAISS vector indices.
View missionEfficient Memory Management for Large Language Model Serving with PagedAttention
Woosuk Kwon · vLLM / UC BerkeleyAuthored the seminal SOSP 2023 paper introducing PagedAttention, inspired by operating system virtual memory paging, allowing non-contiguous KV-cache storage and near-zero memory waste during high-concurrency LLM inference.
Paged cache lookup introduces minor pointer indirection overhead in memory-bandwidth-bound token generation loops.
Context: Evaluated across LLaMA, OPT, and Falcon architectures with continuous batching and multi-query attention.
View missionEnd-to-End Training of Deep Visuomotor Policies
Dr. Sergey Levine · UC Berkeley / Physical IntelligencePublished the foundational Journal of Machine Learning Research paper proving that robotic controllers can map raw camera pixels directly to continuous motor torque voltages using guided policy search.
Direct pixel-to-torque policies are vulnerable to novel environmental lighting and visual background shifts.
Context: Convolutional neural network visuomotor policies operating on physical PR2 robotic systems.
View missionExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5)
Dr. Colin Raffel · University of Toronto / Vector InstituteFirst author of the seminal T5 paper establishing the unified text-to-text paradigm across all NLP tasks, introducing the C4 (Colossal Clean Crawled Corpus) dataset with over 25,000 academic citations.
Encoder-decoder architecture has higher KV-cache memory requirements during generation than pure autoregressive decoders.
Context: Encoder-decoder transformer architecture up to 11B parameters trained on C4 web corpus.
View missionFlash-Decoding: Accelerating Long-Context LLM Inference via Partitioned Attention
Dr. Sasha Rush · Cornell Tech / Hugging FaceCo-developed Flash-Decoding, an inference parallelization algorithm that partitions key-value caches across thread blocks during token generation, delivering up to 8x speedup on 64k+ context sequences.
Requires reduction step across partitions that introduces minor overhead for very short sequence lengths.
Context: CUDA kernels, split-KV cache reduction, integrated into vLLM, TensorRT-LLM, and PyTorch 2.2.
View missionFLUX.1: 12B Parameter Rectified Flow Transformer for High-Fidelity Image Generation
Dr. Robin Rombach · Black Forest LabsArchitected FLUX.1, an open-weights 12B parameter multimodal diffusion transformer utilizing rectified flow matching, achieving unmatched prompt adherence, typography generation, and photographic realism.
12B model footprint requires 24GB+ VRAM or 8-bit/4-bit quantization (NF4) for local inference on consumer hardware.
Context: 12B parameter multimodal Diffusion Transformer (MMDiT), rectified flow matching, rotary positional embeddings.
View missionHigh-Resolution Image Synthesis with Latent Diffusion Models (Stable Diffusion)
Dr. Robin Rombach · Black Forest LabsLead author of the landmark CVPR 2022 paper introducing Latent Diffusion Models (LDMs), shifting diffusion training from raw pixel space into a compressed latent perceptual space, unlocking consumer GPU diffusion synthesis.
Fine spatial details (e.g., distant human hands or miniature typography) can suffer perceptual compression loss in the VAE latent space.
Context: Autoencoder latent space (8x downsampling), cross-attention conditioning, UNet denoising backbone.
View missionHighly Accurate Protein Structure Prediction with AlphaFold
Dr. Demis Hassabis · Google DeepMindSenior author of the landmark Nature 2021 publication demonstrating atomic-accuracy prediction of 3D protein structures from amino acid sequences, resolving a 50-year grand challenge in molecular biology.
Conformational dynamics in intrinsically disordered regions remain challenging without empirical laboratory validation.
Context: Evoformer attention network, Invariant Point Attention (IPA) 3D coordinate frame refinement.
View missionKILT: Benchmarking Knowledge Intensive Language Tasks for Grounded Generation
Dr. Patrick Lewis · CohereDeveloped the KILT benchmark standardizing evaluation across open-domain QA, fact checking, entity linking, and dialogue, measuring both answer generation accuracy and provenance attribution.
Corpus snapshot drift requires periodic index refreshes to keep knowledge-intensive models updated with real-world facts.
Context: Unified Wikipedia corpus snapshot with passage-level attribution targets.
View missionllama.cpp: High-Performance C/C++ Foundation Model Inference Engine
Georgi Gerganov · ggml.aiEngineered a zero-dependency C/C++ LLM inference framework implementing 2-bit through 8-bit integer quantization (GGUF), ARM NEON, Apple Silicon Metal, CUDA, and AVX2 vector intrinsics, powering the modern local AI ecosystem.
Extreme low-bit quantizations (2-bit/3-bit) exhibit perplexity degradation on complex symbolic reasoning tasks.
Context: GGUF quantization format, Llama 2/3, Mistral, Qwen, DeepSeek; pure C/C++ runtime.
View missionManual Backpropagation Triton Kernel Derivations for Parameter-Efficient Cross-Entropy
Daniel Han · Unsloth AIPublished mathematical explanations and Triton code demonstrating how computing gradient chain rules directly inside custom SRAM kernels avoids materializing multi-gigabyte intermediate tensors in PyTorch.
Custom Triton kernels require deep mathematical derivations for each distinct attention and normalization layer.
Context: Triton cross-entropy loss, RoPE embedding kernels, RMSNorm gradient fusion.
View missionMasked Autoencoders Are Scalable Vision Learners (MAE)
Dr. Kaiming He · MIT EECSDeveloped Masked Autoencoders (MAE), proving that masking 75% of image patches and using an asymmetric vision transformer autoencoder allows models to learn rich visual representations self-supervised at massive scale.
Requires substantial pre-training compute and large unlabeled image datasets for optimal self-supervised convergence.
Context: Vision Transformer (ViT-Large/Huge), asymmetric encoder-decoder, 75% random masking.
View missionMastering the Game of Go with Deep Neural Networks and Tree Search (AlphaGo)
Dr. Demis Hassabis · Google DeepMindDirected the historic AlphaGo project that combined deep convolutional value/policy networks with Monte Carlo Tree Search (MCTS) to defeat 18-time world champion Lee Sedol in Go.
Required massive simulated game rollouts; subsequent AlphaZero generalized the principle without human game data.
Context: Self-play reinforcement learning, policy and value convolutional networks, asynchronous MCTS.
View missionMLIR: Multi-Level Intermediate Representation for Heterogeneous AI Accelerators
Dr. Chris Lattner · ModularCo-authored the landmark paper establishing MLIR, a unified compiler framework that bridges high-level machine learning graphs with diverse hardware accelerators (GPUs, TPUs, custom ASICs) through extensible dialects.
Dialect conversions require rigorous domain-specific lowering pipelines for novel specialized silicon.
Context: C++ compiler framework, LLVM backend, dialect ecosystem (Linalg, Affine, Vector, GPU).
View missionModel-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML)
Dr. Chelsea Finn · Stanford UniversityAuthored the landmark ICML 2017 paper creating MAML, a gradient-based meta-learning algorithm that optimizes neural network initializations so that models can learn new tasks from only a few gradient steps.
Second-order Hessian computations impose high memory overhead during meta-training iterations.
Context: Second-order meta-gradient optimization across few-shot classification and robotic continuous control.
View missionMulti-File Codebase Indexing & Semantic Agentic Refactoring
Aman Sanger · Cursor / AnysphereEngineered local embedding and symbol graph pipelines capable of indexing hundred-thousand-line monorepos into contextual embeddings, enabling multi-step agentic software modifications.
Context budgets require strict retrieval pruning in repositories with dense circular dependencies.
Context: Custom tree-sitter chunking, semantic vector index, agentic terminal command execution loop.
View mission