AI research and projects
The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.
A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗
Gemini: A Family of Highly Capable Multimodal Models
Jeff Dean · Google / Google DeepMindCo-led the executive and technical direction of Gemini, designed natively from the ground up to interleave and reason across text, image, video, audio, and code tokens across Google TPU v4 and TPU v5e supercomputing pods.
Interleaved audio-video processing requires aggressive token pruning to sustain real-time serving latency.
Context: Native multimodal transformer trained on TPU v4/v5e clusters with million-token context windows.
View missionGloVe: Global Vectors for Word Representation
Christopher Manning · Stanford UniversityCreated the GloVe log-bilinear word embedding model that combines the advantages of global matrix factorization and local context window methods, capturing linear substructures and semantic relationships in vector space.
Static word vectors conflate multiple distinct polysemous meanings of words into a single point representation.
Context: Trained on Common Crawl 42B and 840B token corpora.
View missionGrandmaster Level in StarCraft II using Multi-Agent Reinforcement Learning (AlphaStar)
Oriol Vinyals · Google DeepMindLed the AlphaStar research team combining imitation learning, multi-agent league training, and deep transformer-based recurrent policies to master StarCraft II—operating under imperfect information, long-horizon real-time strategy, and high-dimensional action spaces.
League diversity must be actively curated with dedicated exploiters to prevent cyclic policy stagnation.
Context: Multi-agent league self-play across 35 days of TPU v3 training per agent with human-like action rate limits.
View missionImageNet Classification with Deep Convolutional Neural Networks (AlexNet)
Geoffrey Hinton · University of TorontoCo-authored the milestone AlexNet paper that crushed the ImageNet competition in 2012 by an unprecedented 10.8% margin, demonstrating the power of GPU-accelerated convolutional networks with ReLU activations and Dropout regularization.
Required heavy data augmentation (cropping, flipping, PCA jittering) to prevent severe overfitting on large parameter weights.
Context: 8-layer CNN trained across two NVIDIA GeForce GTX 580 GPUs on 1.2 million high-resolution images.
View missionImproving Language Understanding by Generative Pre-Training (GPT-1)
Ilya Sutskever · Safe Superintelligence Inc. (SSI)Co-architected the foundational framework establishing that unsupervised autoregressive pre-training on diverse text corpora, followed by discriminative task fine-tuning, produces superior transfer learning across reasoning, classification, and natural language inference benchmarks.
Fine-tuning still required bespoke task heads prior to the zero-shot prompting breakthrough of GPT-2 and GPT-3.
Context: 12-layer decoder-only transformer with masked self-attention heads trained on BooksCorpus.
View missionLearning Representations by Back-Propagating Errors
Geoffrey Hinton · University of TorontoAuthored the foundational 1986 Nature paper demonstrating that backward propagation of error gradients enables multi-layer neural networks to learn internal representations, resolving the perceptron limits and founding modern connectionist AI.
Susceptible to vanishing/exploding gradients in extremely deep architectures prior to residual connections and normalized activations.
Context: Multi-layer perceptrons with generalized delta rule gradient descent.
View missionMastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (AlphaZero)
David Silver · Google DeepMind / UCLCreated AlphaZero, which mastered Go, chess, and shogi starting completely from scratch tabula rasa given only the game rules, crushing world-champion engines Stockfish and Elmo within 24 hours of self-play.
Assumes deterministic, fully observable game dynamics with explicit simulator rules.
Context: Single deep residual neural network updating continuous MCTS action evaluations across 5,000 TPU v1s.
View missionMastering the Game of Go with Deep Neural Networks and Tree Search (AlphaGo)
David Silver · Google DeepMind / UCLLed the AlphaGo project combining deep value networks and policy networks with Monte Carlo Tree Search (MCTS), solving the 2,500-year-old game of Go and defeating 18-time world champion Lee Sedol 4-1 in Seoul.
Initial AlphaGo version relied on 30 million human expert games for supervised pretraining prior to self-play.
Context: 12-layer deep convolutional policy and value networks trained across 48 Google TPUs.
View missionModel-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML)
Pieter Abbeel · UC Berkeley / CovariantFormulated the MAML meta-learning algorithm that optimizes model parameters such that a small number of gradient steps on a tiny amount of new data produces rapid adaptation across vision, regression, and robotics reinforcement learning tasks.
Second-order gradient computations through inner-loop optimization paths impose significant memory and compute overhead.
Context: Deep policy gradients and meta-optimization on Omniglot, MiniImageNet, and simulated robotic locomotion.
View missionMultimodal Datasets: Misogyny, Pornography, and Malign Content in LAION-400M
Abeba Birhane · Mozilla / Trinity College DublinConducted the rigorous empirical safety audit of the LAION-400M multimodal dataset used to train models like Stable Diffusion, exposing widespread presence of non-consensual imagery, racist caricatures, and toxic associations in web-scraped training pairs.
Automated keyword and similarity filtering misses nuanced contextual harms without intensive manual human review.
Context: Multimodal image-text pairs indexed via Common Crawl and CLIP cosine similarity filters.
View missionNeural Machine Translation by Jointly Learning to Align and Translate (Additive Attention)
Yoshua Bengio · Mila - Quebec AI Institute / Université de MontréalCo-authored the milestone paper introducing the attention mechanism into sequence-to-sequence translation, allowing decoders to dynamically attend to relevant input positions and directly inspiring the Transformer architecture.
Recurrent sequential processing precluded massive cluster-scale parallelization.
Context: Bidirectional RNN encoder with Bahdanau additive attention decoder on WMT14.
View missionPractical LoRA & QLoRA Fine-Tuning Parameter Analysis & Benchmarks
Sebastian Raschka · Lightning AIPublished empirical studies benchmarking the impact of rank (r), alpha scaling, target weight modules (Q, K, V vs all linear layers), and 4-bit NormalFloat quantization on model performance across Llama and Mistral.
Aggressive 4-bit quantization can induce subtle degradation on specialized domain syntax without rank compensation.
Context: 7B to 70B parameter models fine-tuned across consumer GPU hardware with LitGPT.
View missionProgress Measures for Grokking via Mechanistic Interpretability
Neel Nanda · Google DeepMindReverse-engineered the exact circular Fourier representation circuits formed when transformers 'grok' modular addition long after overfitting, proving that continuous generalization progresses silently before discontinuous performance jumps.
Toy model arithmetic circuits do not fully capture superposition and polysemanticity present in production-scale language models.
Context: One-layer transformers trained on modular arithmetic tasks with weight decay.
View missionRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Richard Socher · You.comIntroduced Recursive Neural Tensor Networks (RNTN) and the Stanford Sentiment Treebank, establishing how compositional semantic representations can be computed over grammatical parse trees to capture negation and complex sarcasm.
Tree-structured recursive models require syntactic parse trees as inputs, creating parsing error propagation bottlenecks.
Context: Recursive neural tensor networks evaluated on 11,855 parsed sentences from Rotten Tomatoes movie reviews.
View missionReinforcement Learning for Personalized Dynamic Treatment Regimes
Joelle Pineau · Meta FAIR / McGill UniversityPioneered offline batch reinforcement learning formulations optimizing multi-stage clinical interventions for chronic diseases including epilepsy and pediatric cancer, adapting dosages over time while minimizing toxic side effects.
High clinical safety thresholds require provable off-policy evaluation bounds before live patient deployment.
Context: POMDP algorithms evaluated against clinical patient trial trajectories.
View missionScaling Instruction-Finetuned Language Models (Flan-T5 & Flan-PaLM)
Hyung Won Chung · OpenAISpearheaded the research demonstrating that instruction fine-tuning scaled across 1,836 diverse tasks and 540 billion parameters enhances task-general reasoning, chain-of-thought problem solving, and zero-shot performance across every benchmark.
Instruction tuning without careful task balance can degrade pure memorization capacity on open-domain fact retrieval.
Context: Flan-T5 and Flan-PaLM models evaluated on MMLU, Big-Bench, and HumanEval.
View mission