AI research and projects
The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.
A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗
Deep Speech: Scaling up End-to-End Speech Recognition
Dr. Awni Hannun · Apple Machine Learning ResearchFirst author of the landmark Baidu Deep Speech paper, proving that replacing complex phonetic acoustic pipelines with a single recurrent neural network trained end-to-end with CTC achieves state-of-the-art accuracy in noisy real-world speech.
End-to-end recurrent models without attention mechanisms struggle to maintain alignment over multi-minute audio streams.
Context: Bidirectional RNNs, CTC loss, custom GPU parallel prefix beam search decoders.
View missionEfficient Memory Management for Structured Generation in Large Language Models
Dr. Sheng Shen · UC Berkeley BAIRFormulated algorithms for structured JSON and regex-constrained LLM generation that maintain jump-forward finite-state machines without re-allocating KV caches, accelerating structured extraction by 3x.
Complex nested recursive schemas require state graph compilation before first token generation.
Context: Constrained grammar parsing, Outlines/XGrammar integration, LLaMA-2/3.
View missionEvaluating LLM Code Generation with Abstract Syntax Tree Validation
Thorsten Ball · Independent Author & Systems EngineerDeveloped methodologies for benchmarking AI coding assistants using compiler AST validation and deterministic unit test harnesses, identifying syntax failure modes in LLM code diffing.
Syntactic correctness does not guarantee semantic alignment or absence of subtle runtime edge-case bugs.
Context: AST parsing, tree-sitter integration, Claude 3.5 Sonnet, GPT-4o code completion.
View missionHuman-level Play in the Game of Diplomacy with Open-Source CICERO
Dr. Noam Brown · OpenAILead author of the landmark Science 2022 paper creating CICERO, which combined strategic reasoning (piKL planning algorithm) with dialogue generation to negotiate, coordinate, and achieve human-level play in full-game Diplomacy.
Dialogue filtering requires strict truthfulness classifiers to prevent strategic hallucinations that violate game rules.
Context: Strategic value iteration, filtered Monte Carlo dialogue planning, 7-player cooperative-competitive games.
View missionHybrid Computing Using a Neural Network with Dynamic External Memory
Dr. Alex Graves · NNAISENSE / former Google DeepMindPublished the Nature 2016 cover paper introducing Differentiable Neural Computers (DNCs), demonstrating that neural controllers coupled with readable and writable memory matrices can learn complex algorithmic logic and graph navigation.
Complex memory addressing mechanisms induce high computational overhead compared to static Transformer attention.
Context: Differentiable read/write heads, temporal linkage matrices, algorithmic reasoning.
View missionImage-to-Image Translation with Conditional Adversarial Networks (Pix2Pix)
Dr. Phillip Isola · MITAuthored the iconic CVPR 2017 paper defining conditional GANs for pixel-to-pixel translation tasks (edge-to-photo, night-to-day, map-to-aerial), establishing PatchGAN discriminators as standard generative building blocks.
Requires paired training samples, which are costly or impossible to obtain for arbitrary unsupervised real-world domains.
Context: U-Net generator, 70x70 PatchGAN discriminator, paired dataset training.
View missionInteractive Ray Tracing and Neural Radiance Caching Algorithms
Dr. Károly Zsolnai-Fehér · Two Minute Papers / TU WienAuthored peer-reviewed publications in ACM Transactions on Graphics and SIGGRAPH detailing real-time path tracing, importance sampling, and neural radiance cache interpolation.
Radiance cache extrapolation degrades on rapid non-diffuse lighting discontinuities.
Context: CUDA, C++, Monte Carlo integration, neural radiance interpolation.
View missionJudging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Lianmin Zheng · LMSYS Org / UC BerkeleyCo-authored the landmark NeurIPS 2023 paper establishing crowd-sourced, blind pairwise human evaluation (Chatbot Arena) and LLM-as-a-judge benchmarks, standardizing empirical foundation model comparison globally.
Susceptible to verbosity and style bias when using automated LLM evaluators without strict length-penalized rubrics.
Context: Crowdsourced human Elo ratings across 1M+ pairwise conversations and GPT-4 automated judgment.
View missionLearning to Learn with Generative Models of Neural Network Checkpoints
William Peebles · OpenAIAuthored the NeurIPS 2022 paper exploring generative modeling of neural network parameter checkpoints, demonstrating that diffusion models can synthesize functioning weights of performant downstream classifiers.
Permutation symmetry in high-dimensional weight spaces makes scaling to multi-billion parameter checkpoints difficult.
Context: Graph hypernetworks, weight-space diffusion, permutation invariance.
View missionLearning Transferable Visual Models From Natural Language Supervision (CLIP)
Alec Radford · OpenAIAuthored the foundational ICML 2021 paper introducing CLIP, which trained dual vision and text encoders via symmetric cross-entropy contrastive loss on 400M image-text pairs, unlocking robust zero-shot image classification and powering Stable Diffusion.
Fine-grained spatial reasoning, counting, and typographic reading struggle without explicit spatial bounding box supervision.
Context: Vision Transformers (ViT-B/16, ViT-L/14) and ResNet baselines, 400M image-text dataset.
View missionLLaMA: Open and Efficient Foundation Language Models
Guillaume Lample · Mistral AICore co-author of the original Meta LLaMA 7B-65B release, proving that smaller models trained on significantly more tokens than Chinchilla recommendations surpass larger architectures and enable democratized research.
Base pre-trained checkpoints require safety and instruction fine-tuning to prevent conversational misalignment.
Context: SwiGLU activations, RMSNorm, Rotary Embeddings, 1.4T tokens.
View missionLoRA: Low-Rank Adaptation of Large Language Models
Edward Hu · Microsoft ResearchAuthored the foundational ICLR 2022 paper introducing LoRA, which freezes pre-trained model weights and injects trainable rank decomposition matrices into Transformer attention layers, matching full fine-tuning quality with 3x less GPU VRAM.
Adapting non-attention feed-forward layers yields diminishing returns unless higher rank dimensions are budgeted.
Context: GPT-3 175B, RoBERTa, DeBERTa, rank 1-8 adapters with zero additional inference latency after matrix folding.
View missionMixtral of Experts: Sparse Mixture of Experts with Open Weights
Guillaume Lample · Mistral AICo-authored the paper introducing Mixtral 8x7B, a decoder-only model where each layer is composed of 8 feedforward blocks with a router network that selects 2 experts per token, outperforming Llama 2 70B while maintaining the inference speed of a 12B model.
MoE models require substantial VRAM for weight storage despite sparse compute activations during generation.
Context: Sparse MoE, 47B total parameters, 13B active per token, 32k context with RoPE.
View missionPyTorch 2: Dynamic Python Bytecode Compilation and TorchInductor
Horace He · Thinking Systems / former PyTorch CoreCo-authored the ACM ASPLOS 2024 paper detailing TorchDynamo and TorchInductor, which intercept Python frame evaluation bytecodes to capture computational graphs safely and generate fused Triton GPU kernels with 0% code modifications.
Dynamic control flow with non-tensor Python data structures forces graph breaks that require partial eager execution fallback.
Context: TorchDynamo, TorchInductor, OpenAI Triton backend, 160+ benchmark models.
View missionResNet Strikes Back: An Improved Training Procedure in PyTorch
Ross Wightman · Hugging FaceCo-authored the NeurIPS 2021 benchmark study demonstrating that modern training recipes (BCE loss, RandAugment, AdamW) lift standard ResNet-50 accuracy from 76.5% to 80.4% without any architectural changes.
Heavy data augmentations significantly increase training epochs and compute overhead compared to vanilla SGD.
Context: ResNet-50, LamB/AdamW optimizers, repeated augmentations.
View missionSandboxed Code Execution Protocols for Autonomous LLM Agents
Sulaiman Ghori · CognosysDeveloped security isolation protocols utilizing gVisor and WebAssembly to safely execute arbitrary agent-generated code with network boundary controls.
Container cold-start latency introduces 1-2 second delays on initial tool execution invocations.
Context: gVisor, WASM runtime, Linux cgroups, LLM tool dispatch.
View mission