PUBLISHED AI RESEARCH AND PROJECTS

AI research and projects

The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.

Clear all filters

A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗

Showing 97–112 of 149 proofs of work · Kind: research · Newest first
Page 7 of 10
researchChecked Sep 21, 2026

Training Language Models to Follow Instructions with Human Feedback

Dr. John Schulman · Anthropic / Thinking Machines

Led the landmark InstructGPT research demonstrating that applying RLHF with fine-grained human preference data to a 1.3B parameter model produces superior instruction compliance compared to a 175B raw model.

Scope & limitations

Reward models can over-index on output verbosity and confident tone rather than factual accuracy.

Context: Supervised fine-tuning (SFT), Bradley-Terry reward modeling, PPO reinforcement learning over GPT-3.

View mission
researchChecked Sep 21, 2026

Tülu 3: Pushing Frontiers in Open Language Model Post-Training

Nathan Lambert, Ph.D. · Allen Institute for AI (Ai2)

Co-led the Tülu 3 open post-training initiative, releasing complete training datasets, recipes, and checkpoint weights demonstrating open-source parity with proprietary frontier models through RLVR and DPO.

Scope & limitations

Verifiable rewards require deterministic verifiers (math/code compilers), limiting RLVR application in subjective writing.

Context: Llama 3.1 base models, Direct Preference Optimization, Reinforcement Learning from Verifiable Rewards (RLVR).

View mission
researchChecked Sep 21, 2026

π₀ (pi_0): Generalist Physical Intelligence Robot Foundation Model

Dr. Sergey Levine · UC Berkeley / Physical Intelligence

Co-developed π₀, a generalist physical intelligence foundation model trained across diverse robotic embodiments and tasks, demonstrating zero-shot physical manipulation, folding laundry, and assembly.

Scope & limitations

Physical real-world manipulation requires sub-millisecond control loops and collision avoidance safeguards.

Context: Flow-matching vision-action transformer architecture controlling multi-degree-of-freedom robotic arms.

View mission
researchChecked Sep 20, 2026

A Path Towards Autonomous Machine Intelligence (JEPA Architecture Framework)

Dr. Yann LeCun · Meta / New York University

Published the comprehensive theoretical blueprint proposing Joint Embedding Predictive Architecture (JEPA) as an alternative to generative pixel-level or token-level prediction, learning representations in abstract embedding space to enable hierarchical planning.

Scope & limitations

Action-conditional latent dynamics models remain active research topics; translating abstract latent representations back into fine-grained execution paths requires auxiliary decoding heads.

Context: Self-supervised non-contrastive energy-based models; architectural basis for I-JEPA (vision) and V-JEPA (video).

View mission
researchChecked Sep 20, 2026

Attention Is All You Need: The Seminal Transformer Architecture Paper

Aidan Gomez · Cohere

Co-authored the groundbreaking NeurIPS 2017 research paper introducing the Transformer, an architecture dispensing entirely with recurrence and convolutions in favor of multi-head self-attention mechanisms, founding modern LLM technology.

Scope & limitations

Original architecture scaled quadratically O(N^2) with sequence length, prompting subsequent research into grouped-query attention, sparse attention, and linear recurrence hybrids.

Context: 8-head multi-head self-attention, sinusoidal positional encodings, trained on 8 NVIDIA P100 GPUs.

View mission
researchChecked Sep 20, 2026

Bottleneck Transformers for Visual Recognition (BoTNet)

Aravind Srinivas · Perplexity AI

Co-authored BoTNet, an architecture that replaces the final spatial convolutions in ResNet with Multi-Head Self-Attention, achieving 84.7% Top-1 accuracy on ImageNet while substantially improving instance segmentation and object detection on COCO.

Scope & limitations

Quadratic memory complexity in early spatial layers restricts attention blocks to deeper, downsampled feature maps in the network.

Context: ResNet backbone with Multi-Head Self-Attention bottleneck blocks, TPU v3 accelerators, 2D relative position encodings.

View mission
researchChecked Sep 20, 2026

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Dr. Jason Wei · OpenAI

Authored the seminal NeurIPS 2022 research paper demonstrating that generating intermediate reasoning steps dramatically boosts LLM performance on arithmetic, symbolic, and multi-hop reasoning tasks.

Scope & limitations

Chain-of-thought gains exhibit emergence only at significant model scale (~50B+ parameters); smaller models can generate superficially coherent rationales that lead to incorrect arithmetic conclusions.

Context: PaLM 540B, GPT-3 175B, GSM8K, SVAMP, BIG-bench reasoning tasks.

View mission
researchChecked Sep 20, 2026

ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

Dr. Omar Khattab · Stanford University

Invented ColBERT, a neural retrieval model utilizing late interaction over token-level representations rather than collapsing documents into a single dense vector, achieving cross-encoder retrieval quality with millisecond vector search latency.

Scope & limitations

Multi-vector token index storage requires larger RAM and disk allocations than single-vector embedding stores unless aggressive centroid quantization (ColBERTv2 / PLAID) is applied.

Context: Token-level BERT embeddings, MaxSim operator, PLAID index compression.

View mission
researchChecked Sep 20, 2026

Datasheets for Datasets: Standardized Transparency Protocol

Dr. Timnit Gebru · Distributed AI Research Institute (DAIR)

Co-authored the CACM benchmark paper establishing 'Datasheets for Datasets', standardizing documentation of motivation, composition, collection process, preprocessing, and recommended uses across machine learning training datasets.

Scope & limitations

Compliance is self-reported by dataset creators; verification requires independent audits and automated dataset inspection tooling.

Context: Public policy, dataset governance, and risk mitigation framework adopted across major ML labs.

View mission
researchChecked Sep 20, 2026

De-anonymization and Privacy Risks in Large-Scale Data Platforms

Dr. Arvind Narayanan · Princeton University

Proved that high-dimensional datasets (Netflix Prize, social graphs) can be re-identified with high probability using sparse auxiliary information, refuting the concept of 'anonymized' behavioral data in machine learning training sets.

Scope & limitations

Modern differential privacy mechanisms mitigate some vulnerability but introduce bounded utility trade-offs in downstream model training.

Context: High-dimensional matrix factorizations, graph matching algorithms, differential privacy formulations.

View mission
researchChecked Sep 20, 2026

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines

Dr. Omar Khattab · Stanford University

Authored the DSPy Assertions paradigm, introducing computational assertions and suggestions directly into declarative LM graphs, enabling pipelines to self-correct upon constraint failure without manual prompt rewriting.

Scope & limitations

Backtracking loops require hard iteration ceilings to avoid infinite retry loops when models encounter unresolvable logical contradictions.

Context: Dynamic backtracking execution graph in Python, automated metric validation.

View mission
researchChecked Sep 20, 2026

Emergent Abilities of Large Language Models

Dr. Jason Wei · OpenAI

Published the influential study demonstrating that certain capabilities (multi-step arithmetic, translation, symbolic reasoning) are not present in smaller models and emerge non-linearly only once compute and parameter scales cross critical thresholds.

Scope & limitations

Metric non-linearities (such as exact-string match versus continuous token log-likelihood) can exaggerate the appearance of abrupt phase transitions.

Context: Empirical analysis across PaLM, GPT-3, Gopher, Chinchilla, and LaMDA across dozens of benchmark evaluations.

View mission
researchChecked Sep 20, 2026

Evaluating the Fairness and Accuracy of Algorithmic Risk Assessment Instruments

Dr. Arvind Narayanan · Princeton University

Authored the classic tutorial and survey demonstrating the mathematical impossibility of satisfying three common notions of algorithmic fairness simultaneously, proving the need for explicit ethical trade-offs in automated scoring.

Scope & limitations

Mathematical trade-off proofs assume stationary data distributions and cannot resolve normative political disagreements regarding priority.

Context: COMPAS recidivism scores, credit risk models, calibration vs error-rate parity statistical constraints.

View mission
researchChecked Sep 20, 2026

Fast Transformer Decoding: One Write-Head is All You Need (Multi-Query Attention)

Noam Shazeer · Google DeepMind

Authored the foundational paper introducing Multi-Query Attention (MQA), where all attention heads share a single key and value head. This drastically slashes memory bandwidth consumption during autoregressive generation by up to 10x with negligible accuracy drop.

Scope & limitations

Slightly reduces representational capacity in small model regimes compared to standard multi-head attention before being mitigated by modern GQA configurations.

Context: Decoupled query/key/value projection matrices; foundation for Grouped-Query Attention (GQA) across modern LLMs like Llama 3 and Mistral.

View mission
researchChecked Sep 20, 2026

GLU Variants Improve Transformer (SwiGLU Activation Function)

Noam Shazeer · Google DeepMind

Formulated and benchmarked Gated Linear Unit variants (GLU, Bilinear, ReGLU, GEGLU, SwiGLU) for transformer feedforward layers, demonstrating that SwiGLU consistently outperforms standard ReLU and GELU activations.

Scope & limitations

Requires three weight matrices in feedforward blocks instead of two, requiring adjustments to intermediate hidden dimensions to maintain equivalent parameter counts.

Context: Swish-gated linear projections: SwiGLU(x) = Swish(xW) * (xV).

View mission
researchChecked Sep 20, 2026

ImageNet: A Large-Scale Hierarchical Image Database

Dr. Fei-Fei Li · Stanford University / World Labs

Spearheaded the creation and curation of ImageNet, an ontological visual dataset containing over 14 million annotated images across 20,000 synsets, which launched the annual ImageNet Large Scale Visual Recognition Challenge (ILSVRC) and ignited AlexNet and deep learning.

Scope & limitations

Class distributions reflected internet imagery availability at the time; subsequent audits addressed cultural and geographical representation biases.

Context: WordNet hierarchy mapping, Amazon Mechanical Turk crowdsourced quality verification protocol.

View mission
Showing 97–112 of 149 proofs of work