PUBLISHED AI RESEARCH AND PROJECTS

AI research and projects

The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.

A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗

Showing 81–96 of 567 proofs of work · Newest first
Page 6 of 36
researchChecked Sep 23, 2026

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML)

Pieter Abbeel · UC Berkeley / Covariant

Formulated the MAML meta-learning algorithm that optimizes model parameters such that a small number of gradient steps on a tiny amount of new data produces rapid adaptation across vision, regression, and robotics reinforcement learning tasks.

Scope & limitations

Second-order gradient computations through inner-loop optimization paths impose significant memory and compute overhead.

Context: Deep policy gradients and meta-optimization on Omniglot, MiniImageNet, and simulated robotic locomotion.

View mission
researchChecked Sep 23, 2026

Multimodal Datasets: Misogyny, Pornography, and Malign Content in LAION-400M

Abeba Birhane · Mozilla / Trinity College Dublin

Conducted the rigorous empirical safety audit of the LAION-400M multimodal dataset used to train models like Stable Diffusion, exposing widespread presence of non-consensual imagery, racist caricatures, and toxic associations in web-scraped training pairs.

Scope & limitations

Automated keyword and similarity filtering misses nuanced contextual harms without intensive manual human review.

Context: Multimodal image-text pairs indexed via Common Crawl and CLIP cosine similarity filters.

View mission
researchChecked Sep 23, 2026

Neural Machine Translation by Jointly Learning to Align and Translate (Additive Attention)

Yoshua Bengio · Mila - Quebec AI Institute / Université de Montréal

Co-authored the milestone paper introducing the attention mechanism into sequence-to-sequence translation, allowing decoders to dynamically attend to relevant input positions and directly inspiring the Transformer architecture.

Scope & limitations

Recurrent sequential processing precluded massive cluster-scale parallelization.

Context: Bidirectional RNN encoder with Bahdanau additive attention decoder on WMT14.

View mission
researchChecked Sep 23, 2026

Practical LoRA & QLoRA Fine-Tuning Parameter Analysis & Benchmarks

Sebastian Raschka · Lightning AI

Published empirical studies benchmarking the impact of rank (r), alpha scaling, target weight modules (Q, K, V vs all linear layers), and 4-bit NormalFloat quantization on model performance across Llama and Mistral.

Scope & limitations

Aggressive 4-bit quantization can induce subtle degradation on specialized domain syntax without rank compensation.

Context: 7B to 70B parameter models fine-tuned across consumer GPU hardware with LitGPT.

View mission
researchChecked Sep 23, 2026

Progress Measures for Grokking via Mechanistic Interpretability

Neel Nanda · Google DeepMind

Reverse-engineered the exact circular Fourier representation circuits formed when transformers 'grok' modular addition long after overfitting, proving that continuous generalization progresses silently before discontinuous performance jumps.

Scope & limitations

Toy model arithmetic circuits do not fully capture superposition and polysemanticity present in production-scale language models.

Context: One-layer transformers trained on modular arithmetic tasks with weight decay.

View mission
researchChecked Sep 23, 2026

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Richard Socher · You.com

Introduced Recursive Neural Tensor Networks (RNTN) and the Stanford Sentiment Treebank, establishing how compositional semantic representations can be computed over grammatical parse trees to capture negation and complex sarcasm.

Scope & limitations

Tree-structured recursive models require syntactic parse trees as inputs, creating parsing error propagation bottlenecks.

Context: Recursive neural tensor networks evaluated on 11,855 parsed sentences from Rotten Tomatoes movie reviews.

View mission
researchChecked Sep 23, 2026

Reinforcement Learning for Personalized Dynamic Treatment Regimes

Joelle Pineau · Meta FAIR / McGill University

Pioneered offline batch reinforcement learning formulations optimizing multi-stage clinical interventions for chronic diseases including epilepsy and pediatric cancer, adapting dosages over time while minimizing toxic side effects.

Scope & limitations

High clinical safety thresholds require provable off-policy evaluation bounds before live patient deployment.

Context: POMDP algorithms evaluated against clinical patient trial trajectories.

View mission
implementationChecked Sep 23, 2026

SafeTensors: Fast, Secure Tensor Storage for Deep Learning

Julien Chaumond · Hugging Face

Spearheaded the creation and adoption of SafeTensors, replacing insecure Python pickle files with zero-copy memory-mapped deserialization that prevents arbitrary code execution and boosts model loading speeds 2-10x.

Scope & limitations

Requires strict contiguous memory layout incompatible with raw arbitrary python object serialization.

Context: Rust-based tensor serialization format adopted universally across Hugging Face, PyTorch, and vLLM.

View mission
implementationChecked Sep 23, 2026

Scale AI Data Engine for Supervised Fine-Tuning & RLHF at Scale

Alexandr Wang · Scale AI

Engineered human-in-the-loop expert annotation pipelines generating multi-million token pairs of complex instruction following, code review, and preference datasets powering the training of leading open and commercial models.

Scope & limitations

Human expert verification costs scale super-linearly with problem complexity in post-doctoral mathematics and legal reasoning.

Context: Distributed annotation platforms with automated inter-annotator agreement metrics and synthetic validation loops.

View mission
researchChecked Sep 23, 2026

Scaling Instruction-Finetuned Language Models (Flan-T5 & Flan-PaLM)

Hyung Won Chung · OpenAI

Spearheaded the research demonstrating that instruction fine-tuning scaled across 1,836 diverse tasks and 540 billion parameters enhances task-general reasoning, chain-of-thought problem solving, and zero-shot performance across every benchmark.

Scope & limitations

Instruction tuning without careful task balance can degrade pure memorization capacity on open-domain fact retrieval.

Context: Flan-T5 and Flan-PaLM models evaluated on MMLU, Big-Bench, and HumanEval.

View mission
researchChecked Sep 23, 2026

Scaling Laws for Neural Language Models

Dario Amodei · Anthropic

Discovered empirical power-law relationships between transformer performance (cross-entropy loss) and model parameter count, dataset size, and compute budget spanning over six orders of magnitude, providing the blueprint for modern frontier foundation model investment.

Scope & limitations

Scaling power laws hold primarily for pretraining loss; downstream task sample efficiency and multi-step reasoning require structured inference-time compute scaling.

Context: Autoregressive language models scaled from 10^3 to 10^9 non-embedding parameters across WebText.

View mission
implementationChecked Sep 23, 2026

SEAL Leaderboards: Standardized Private Evals for Frontier LLM Benchmarking

Alexandr Wang · Scale AI

Created the SEAL evaluation platform providing contamination-free, expert-curated benchmark suites testing advanced programming (Coding-GSM), mathematical rigor, and adversarial safety to combat public leaderboard gaming.

Scope & limitations

Requires ongoing prompt refresh cycles to prevent indirect test data leakage into web-crawled pretraining corpora.

Context: Private held-out dynamic evaluation testbeds continuously updated by human domain experts.

View mission
researchChecked Sep 23, 2026

Sequence to Sequence Learning with Neural Networks

Ilya Sutskever · Safe Superintelligence Inc. (SSI)

Pioneered the multi-layered Long Short-Term Memory (LSTM) encoder-decoder architecture mapping arbitrary sequence lengths to variable output sequences. Revolutionized machine translation, conversational AI, and established the direct conceptual architecture upon which transformers and autoregressive LLMs were built.

Scope & limitations

Recurrent execution bottlenecked parallelization across large cluster topologies before self-attention mechanisms emerged.

Context: Deep LSTM networks, WMT14 English-to-French benchmark, 160,000 input vocabulary.

View mission
researchChecked Sep 23, 2026

Show and Tell: A Neural Image Caption Generator

Oriol Vinyals · Google DeepMind

Pioneered the integration of deep convolutional neural networks (CNNs) for vision encoding with recurrent language models (RNNs) for text generation, producing the first end-to-end neural image captioning pipeline.

Scope & limitations

Susceptible to hallucinating objects based on strong linguistic prior frequencies rather than verified visual presence.

Context: GoogLeNet CNN feature extractor combined with LSTM language decoder on MSCOCO dataset.

View mission
researchChecked Sep 23, 2026

Sparks of Artificial General Intelligence: Early experiments with GPT-4

Sebastien Bubeck · Microsoft Research

Authored the 154-page seminal investigation demonstrating that early non-multimodal GPT-4 exhibited emergent general intelligence across mathematics, coding, law, vision-via-code, psychology, and medicine without domain-specific fine-tuning.

Scope & limitations

Highlighted critical remaining flaws including lack of long-term planning, working memory degradation, and blind confidence in false premises.

Context: Unconstrained early GPT-4 base model checkpoint evaluated across novel out-of-distribution reasoning probes.

View mission
explanationChecked Sep 23, 2026

Stanford CS224N: Natural Language Processing with Deep Learning

Christopher Manning · Stanford University

Created and delivered the globally benchmarked Stanford university curriculum educating over one million students online on word vectors, recurrent networks, attention mechanisms, transformers, and modern large language models.

Scope & limitations

Curriculum must be overhauled annually to keep pace with rapid shifts in foundation model post-training and inference techniques.

Context: PyTorch deep learning pipelines spanning pretraining, fine-tuning, and self-attention interpretability.

View mission
Showing 81–96 of 567 proofs of work