AI research and projects
The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.
A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗
A Neural Probabilistic Language Model
Yoshua Bengio · Mila - Quebec AI Institute / Université de MontréalIntroduced the seminal concept of learning continuous distributed word feature vectors alongside statistical language modeling, defeating the curse of dimensionality and establishing modern neural language modeling.
Computational bottleneck in softmax normalization over large vocabularies prior to hierarchical softmax and negative sampling.
Context: Feed-forward neural network sharing distributed representation layers across context words.
View missionAdversarial Attacks on Neural Networks in Physical and Digital Environments
Dawn Song · UC BerkeleyDemonstrated that adversarial perturbations crafted via gradient-based optimization survive physical-world transformations (such as camera capture, lighting variations, and printing), exposing critical safety vulnerabilities in autonomous perception systems.
Defending against physical attacks often degrades clean-sample classification accuracy on subtle visual nuances.
Context: Inception and ResNet vision models subjected to physical adversarial stickers and perturbations.
View missionAI Feynman: A Physics-Inspired Method for Symbolic Regression
Max Tegmark · MIT / Future of Life InstituteDeveloped AI Feynman, a recursive neural network algorithm combining dimensional analysis, symmetry detection, and polynomial decomposition to discover exact symbolic mathematical laws from raw scientific experimental data.
Exponential complexity scaling in high-dimensional systems without clear separability or symmetries.
Context: Neural networks trained to identify translational, rotational, and scaling invariants across 100 equations from the Feynman Lectures on Physics.
View missionAir Street Capital AI Spinout & Tech Bio Investment Playbook
Nathan Benaich · Air Street CapitalCreated the venture evaluation blueprint helping top European and American academic research spinouts commercialize foundation models in biology, materials discovery, and physics simulations.
TechBio AI models require expensive wet-lab validation cycles that lag in silico prediction speeds.
Context: Deep learning models applied to protein folding, molecular generative dynamics, and automated wet lab integrations.
View missionAlgorithmic Injustice: A Relational Ethics Approach to AI Systems
Abeba Birhane · Mozilla / Trinity College DublinFormulated the foundational relational cognitive framework critiquing the reduction of human social complexity into static statistical classification matrices, demonstrating how predictive AI disproportionately harms marginalized populations.
Requires regulatory and legislative enforcement to compel commercial organizations to adopt rigorous external auditing.
Context: Applied across algorithmic criminal sentencing, hiring filters, and welfare fraud scoring systems.
View missionAnthropic Responsible Scaling Policy (RSP) Framework Version 1.0 & 2.0
Daniela Amodei · AnthropicFormulated the industry-standard risk management framework linking model capability thresholds (ASL-1 through ASL-4) directly to mandatory physical cybersecurity controls, red-teaming verifications, and operational shutdown protocols prior to model deployment.
Requires continuous empirical evaluation pipelines to reliably detect latent dangerous capabilities in reasoning models before public exposure.
Context: Governance protocol spanning internal cluster security, model weight custody, and chemical/biological risk mitigation across Claude foundation models.
View missionARC Autonomous Replication & Threat Evaluation Framework (METR)
Paul Christiano · Alignment Research Center (ARC)Created the standardized red-teaming benchmark testing whether frontier AI models possess autonomous cyber-offense capabilities, resource acquisition, self-replication, or model weight exfiltration skills.
Agent capabilities evolve rapidly with multi-agent orchestration, requiring continuous suite updates.
Context: Sandboxed agentic execution environments testing GPT-4, Claude 3, and open weights models.
View missionBuild a Large Language Model (From Scratch): Complete PyTorch Implementation
Sebastian Raschka · Lightning AICreated the canonical open-source educational repository teaching engineers how to code, train, and fine-tune a complete GPT-style transformer from scratch in pure PyTorch without external high-level libraries.
Designed primarily for educational transparency; industrial scale multi-node training requires distributed FSDP/Megatron wrappers.
Context: Modular Python/PyTorch codebase implementing multi-head self-attention, BPE tokenization, causal masking, and instruction alignment.
View missionChain-of-Thought Prompting Elicits Reasoning in Large Language Models
Hyung Won Chung · OpenAICo-authored the foundational paper discovering that generating step-by-step reasoning tokens before the final answer dramatically unlocks multi-step arithmetic, commonsense, and symbolic reasoning in foundation models.
Chain-of-thought benefits appear as an emergent property only in models with sufficient scale (~100B+ parameters or instruction-tuned compact models).
Context: PaLM 540B, GPT-3 175B tested on GSM8K math benchmark and SVAMP.
View missionChatGPT Production Launch & Reinforcement Learning from Human Feedback (RLHF) Scaling
Mira Murati · Independent AI Research LabDirected the technical conversion of GPT-3.5 into ChatGPT via conversational RLHF, dialog-state conditioning, and distributed inference optimization that scaled to 100 million active users in two months with high system availability.
Early versions suffered from sycophancy and hallucinations under adversarial jailbreak prompts, necessitating iterative red-teaming.
Context: PPO reinforcement learning on InstructGPT checkpoints deployed across large-scale NVIDIA A100 GPU clusters.
View missionConfidential Computing & Privacy-Preserving LLM Inference Architectures
Dawn Song · UC BerkeleyEngineered secure enclave and hardware-assisted cryptographic protocols ensuring user prompt embeddings and model weights remain provably encrypted during remote cloud GPU inference.
Enclave memory constraints introduce bandwidth overhead during large context-window KV cache allocation.
Context: NVIDIA H100 confidential computing enclaves and AMD SEV-SNP virtual machine attestation.
View missionConstitutional AI: Harmlessness from AI Feedback (RLAIF)
Dario Amodei · AnthropicIntroduced the Constitutional AI paradigm enabling models to critique, revise, and align their own behavioral outputs according to written principles without human-labeled toxicity feedback. Established the foundational technique powering Claude's red-teaming resilience.
Constitution design must be continuously audited for latent political biases, conflicting clauses, and nuanced real-world legal edge cases.
Context: Frontier autoregressive transformer checkpoints trained via self-critique, Chain of Thought, and Reinforcement Learning from AI Feedback (RLAIF).
View missionDeep Reinforcement Learning from Human Preferences (RLHF)
Paul Christiano · Alignment Research Center (ARC)Authored the foundational paper establishing that training deep neural policies via pairwise human preference reward models allows agents to master complex tasks (Atari games and robotic backflips) without hard-coded programmatic reward functions.
Reward models can be gamified by agents via reward hacking and sycophantic responses.
Context: PPO actor-critic architectures and learned Bradley-Terry reward models.
View missionDota 2 with Large Scale Deep Reinforcement Learning (OpenAI Five)
Greg Brockman · OpenAILed the engineering and systems team that built OpenAI Five, scaling Proximal Policy Optimization (PPO) across 128,000 CPU cores and 256 GPUs playing 180 years of game simulation daily, defeating reigning world champion esports team OG.
Required heavy reward engineering and hero draft restrictions to converge on winning policies in complex game spaces.
Context: 4,096-unit single-layer LSTM processing continuous state-action spaces with 20,000-step time horizons.
View missionEnterprise Alignment & Secure Frontier Deployment Standards
Daniela Amodei · AnthropicArchitected Anthropic's enterprise data confidentiality guarantees, zero-data-retention training policies, and SOC2 Type II compliance frameworks adopted by major healthcare, financial, and legal institutions worldwide.
Enterprise zero-retention mandates preclude autonomous model feedback loop improvements from raw user traffic.
Context: Claude Enterprise platform on AWS Bedrock and Google Cloud Vertex AI.
View missionEureka: Human-Level Reward Design via Coding Large Language Models
Anima Anandkumar · California Institute of Technology (Caltech)Created Eureka, an autonomous algorithm leveraging frontier coding LLMs to synthesize, evaluate, and iteratively refine complex reward algorithms for reinforcement learning, outperforming human expert-designed rewards across 29 dexterous robot manipulation tasks.
Requires fast GPU simulation environments to provide high-throughput performance feedback to the outer-loop LLM.
Context: GPT-4 reward coding loops interacting with NVIDIA Isaac Gym GPU physics simulations.
View mission