AI research and projects
The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.
A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗
Flash-Decoding: Accelerating Long-Context LLM Inference via Partitioned Attention
Dr. Sasha Rush · Cornell Tech / Hugging FaceCo-developed Flash-Decoding, an inference parallelization algorithm that partitions key-value caches across thread blocks during token generation, delivering up to 8x speedup on 64k+ context sequences.
Requires reduction step across partitions that introduces minor overhead for very short sequence lengths.
Context: CUDA kernels, split-KV cache reduction, integrated into vLLM, TensorRT-LLM, and PyTorch 2.2.
View missionHigh-Resolution Image Synthesis with Latent Diffusion Models (Stable Diffusion)
Dr. Robin Rombach · Black Forest LabsLead author of the landmark CVPR 2022 paper introducing Latent Diffusion Models (LDMs), shifting diffusion training from raw pixel space into a compressed latent perceptual space, unlocking consumer GPU diffusion synthesis.
Fine spatial details (e.g., distant human hands or miniature typography) can suffer perceptual compression loss in the VAE latent space.
Context: Autoencoder latent space (8x downsampling), cross-attention conditioning, UNet denoising backbone.
View missionHighly Accurate Protein Structure Prediction with AlphaFold
Dr. Demis Hassabis · Google DeepMindSenior author of the landmark Nature 2021 publication demonstrating atomic-accuracy prediction of 3D protein structures from amino acid sequences, resolving a 50-year grand challenge in molecular biology.
Conformational dynamics in intrinsically disordered regions remain challenging without empirical laboratory validation.
Context: Evoformer attention network, Invariant Point Attention (IPA) 3D coordinate frame refinement.
View missionKILT: Benchmarking Knowledge Intensive Language Tasks for Grounded Generation
Dr. Patrick Lewis · CohereDeveloped the KILT benchmark standardizing evaluation across open-domain QA, fact checking, entity linking, and dialogue, measuring both answer generation accuracy and provenance attribution.
Corpus snapshot drift requires periodic index refreshes to keep knowledge-intensive models updated with real-world facts.
Context: Unified Wikipedia corpus snapshot with passage-level attribution targets.
View missionMasked Autoencoders Are Scalable Vision Learners (MAE)
Dr. Kaiming He · MIT EECSDeveloped Masked Autoencoders (MAE), proving that masking 75% of image patches and using an asymmetric vision transformer autoencoder allows models to learn rich visual representations self-supervised at massive scale.
Requires substantial pre-training compute and large unlabeled image datasets for optimal self-supervised convergence.
Context: Vision Transformer (ViT-Large/Huge), asymmetric encoder-decoder, 75% random masking.
View missionMastering the Game of Go with Deep Neural Networks and Tree Search (AlphaGo)
Dr. Demis Hassabis · Google DeepMindDirected the historic AlphaGo project that combined deep convolutional value/policy networks with Monte Carlo Tree Search (MCTS) to defeat 18-time world champion Lee Sedol in Go.
Required massive simulated game rollouts; subsequent AlphaZero generalized the principle without human game data.
Context: Self-play reinforcement learning, policy and value convolutional networks, asynchronous MCTS.
View missionMLIR: Multi-Level Intermediate Representation for Heterogeneous AI Accelerators
Dr. Chris Lattner · ModularCo-authored the landmark paper establishing MLIR, a unified compiler framework that bridges high-level machine learning graphs with diverse hardware accelerators (GPUs, TPUs, custom ASICs) through extensible dialects.
Dialect conversions require rigorous domain-specific lowering pipelines for novel specialized silicon.
Context: C++ compiler framework, LLVM backend, dialect ecosystem (Linalg, Affine, Vector, GPU).
View missionModel-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML)
Dr. Chelsea Finn · Stanford UniversityAuthored the landmark ICML 2017 paper creating MAML, a gradient-based meta-learning algorithm that optimizes neural network initializations so that models can learn new tasks from only a few gradient steps.
Second-order Hessian computations impose high memory overhead during meta-training iterations.
Context: Second-order meta-gradient optimization across few-shot classification and robotic continuous control.
View missionOpen X-Embodiment: Robot Learning Datasets and Generalist Policies (RT-X)
Dr. Chelsea Finn · Stanford UniversityCo-led the global Open X-Embodiment consortium uniting 33 academic laboratories across 22 robot types to train RT-X, the world's largest open multi-embodiment robotic policy.
Cross-robot embodiment generalization degrades when sensor modalities or physical gripper geometries differ substantially.
Context: 1M+ robotic trajectory demonstrations across Franka, UR5, Aloha, and mobile manipulators.
View missionParameter-Efficient Transfer Learning for NLP (Foundational PEFT Adapters)
Dr. Colin Raffel · University of Toronto / Vector InstituteCo-authored the landmark ICML 2019 paper introducing lightweight adapter modules injected into transformer layers, proving models could learn new tasks with less than 3% added parameters.
Serial adapter bottlenecks add minor inference latency compared to re-parameterization techniques like LoRA.
Context: Bottleneck projection adapters inserted after multi-head attention and feedforward layers.
View missionProximal Policy Optimization Algorithms (PPO)
Dr. John Schulman · Anthropic / Thinking MachinesInvented PPO, a clipped objective reinforcement learning algorithm that maintains policy stability during stochastic gradient descent, which became the global algorithmic standard for aligning LLMs via RLHF.
Hyperparameter sensitivity to KL penalty coefficients and reward model reward hacking.
Context: Clipped surrogate objective, generalized advantage estimation (GAE), actor-critic networks.
View missionRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Dr. Patrick Lewis · CohereAuthored the foundational NeurIPS 2020 paper defining Retrieval-Augmented Generation (RAG), introducing end-to-end differentiable architectures that ground generative models against external non-parametric neural indices.
Early RAG formulated single-step retrieval; complex multi-hop reasoning requires iterative or agentic retrieval loops.
Context: DPR (Dense Passage Retriever) coupled with BART sequence-to-sequence generation over Wikipedia index.
View missionRewardBench: Evaluating Reward Models for Language Model Alignment
Nathan Lambert, Ph.D. · Allen Institute for AI (Ai2)Created RewardBench, the industry's primary benchmark for evaluating reward models and preference classifiers across chat capabilities, reasoning, and adversarial safety edge cases.
Static evaluation test sets face saturation as training data mixtures incorporate benchmark distributions.
Context: Evaluation suite assessing over 100 reward models across Chat, Reasoning, and Safety axes.
View missionSimCSE: Simple Contrastive Learning of Sentence Embeddings
Dr. Danqi Chen · Princeton UniversityCreated SimCSE, an unsupervised contrastive learning framework that uses standard dropout as data augmentation to learn state-of-the-art sentence embeddings, advancing semantic retrieval without labeled pairs.
Sentence embeddings compress entire paragraphs into single fixed-size vectors, losing granular token-level context.
Context: BERT and RoBERTa backends with contrastive temperature scaling.
View missionSmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Dr. Song Han · MIT / HAN LabInvented SmoothQuant, an algorithm that mathematically splits quantization difficulty between activations and weights via per-channel scaling transformations, unlocking 8-bit weight and activation (W8A8) tensor-core acceleration.
Requires support for per-token dynamic scaling or fixed calibration scale tensors in runtime engines.
Context: INT8 matrix multiplications on NVIDIA Ampere/Ada Lovelace tensor cores.
View missionThe Hardware Lottery: Algorithmic Stagnation and Hardware Alignment
Dr. Sarah Hooker · Cohere For AIAuthored the CACM article formalizing the 'hardware lottery', demonstrating how deep learning architectures win not on fundamental superiority but on whether they fit contemporary GPU and accelerator architectures.
Qualitative and historical systems critique; guiding architectural paradigms rather than releasing a standalone model.
Context: Historical analysis of neural networks, specialized ASICs, and modern transformer tensor architectures.
View mission