PUBLISHED AI RESEARCH AND PROJECTS

AI research and projects

The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.

A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗

Showing 241–256 of 567 proofs of work · Newest first
Page 16 of 36
implementationChecked Sep 22, 2026

LangChain and LLM Agent Notebook Implementations

Sam Witteveen · Red Dragon AI

Authored dozens of open-source Jupyter notebooks with thousands of stars demonstrating tool integration, multi-agent debate, and local RAG retrieval pipelines.

Scope & limitations

Notebook workflows require refactoring into stateless services for production high-concurrency environments.

Context: LangChain, ChromaDB, Hugging Face Transformers, Ollama.

View mission
implementationChecked Sep 22, 2026

LangGraph and CrewAI Production Agent Blueprints

Kris (All About AI) · All About AI

Authored and open-sourced production agent starter repositories with thousands of stars, implementing tool-calling loops, document retrieval, and structured JSON output schemas.

Scope & limitations

Asynchronous task execution requires careful error-handling proxies around flaky external web scrapers.

Context: Python, FastAPI, LangGraph, Ollama, OpenAI Assistant API.

View mission
implementationChecked Sep 22, 2026

Large-Scale Distributed Pre-Training and Sharded Tensor Parallelism

Timothée Lacroix · Mistral AI

Architected the cluster infrastructure utilizing Megatron-LM and custom NCCL collective communication topologies that pre-trained the Mixtral family across thousands of H100 GPUs with near-linear scaling efficiency.

Scope & limitations

Inter-node all-to-all communications in large MoE routing can bottleneck throughput on non-InfiniBand cloud interconnects.

Context: H100 NVLink clusters, InfiniBand HDR/NDR fabric, FP8 mixed precision.

View mission
researchChecked Sep 22, 2026

Learning to Learn with Generative Models of Neural Network Checkpoints

William Peebles · OpenAI

Authored the NeurIPS 2022 paper exploring generative modeling of neural network parameter checkpoints, demonstrating that diffusion models can synthesize functioning weights of performant downstream classifiers.

Scope & limitations

Permutation symmetry in high-dimensional weight spaces makes scaling to multi-billion parameter checkpoints difficult.

Context: Graph hypernetworks, weight-space diffusion, permutation invariance.

View mission
researchChecked Sep 22, 2026

Learning Transferable Visual Models From Natural Language Supervision (CLIP)

Alec Radford · OpenAI

Authored the foundational ICML 2021 paper introducing CLIP, which trained dual vision and text encoders via symmetric cross-entropy contrastive loss on 400M image-text pairs, unlocking robust zero-shot image classification and powering Stable Diffusion.

Scope & limitations

Fine-grained spatial reasoning, counting, and typographic reading struggle without explicit spatial bounding box supervision.

Context: Vision Transformers (ViT-B/16, ViT-L/14) and ResNet baselines, 400M image-text dataset.

View mission
researchChecked Sep 22, 2026

LLaMA: Open and Efficient Foundation Language Models

Guillaume Lample · Mistral AI

Core co-author of the original Meta LLaMA 7B-65B release, proving that smaller models trained on significantly more tokens than Chinchilla recommendations surpass larger architectures and enable democratized research.

Scope & limitations

Base pre-trained checkpoints require safety and instruction fine-tuning to prevent conversational misalignment.

Context: SwiGLU activations, RMSNorm, Rotary Embeddings, 1.4T tokens.

View mission
researchChecked Sep 22, 2026

LoRA: Low-Rank Adaptation of Large Language Models

Edward Hu · Microsoft Research

Authored the foundational ICLR 2022 paper introducing LoRA, which freezes pre-trained model weights and injects trainable rank decomposition matrices into Transformer attention layers, matching full fine-tuning quality with 3x less GPU VRAM.

Scope & limitations

Adapting non-attention feed-forward layers yields diminishing returns unless higher rank dimensions are budgeted.

Context: GPT-3 175B, RoBERTa, DeBERTa, rank 1-8 adapters with zero additional inference latency after matrix folding.

View mission
implementationChecked Sep 22, 2026

LoRA: Official Parameter-Efficient Open-Source PyTorch Implementation

Edward Hu · Microsoft Research

Created and open-sourced the original PyTorch LoRA package providing drop-in `nn.Linear`, `nn.Embedding`, and `nn.Conv2d` layers with automated weight merging, forming the foundation of modern Hugging Face PEFT.

Scope & limitations

Dynamic multi-tenant serving requires runtime adapter switching kernels (e.g., S-LoRA/Punica) to avoid memory fragmentation.

Context: PyTorch 1.8+, Hugging Face integration, scalable rank scaling factor alpha.

View mission
explanationChecked Sep 22, 2026

Making Deep Learning Go Brrrr: Performance Engineering and GPU Kernel Optimization

Horace He · Thinking Systems / former PyTorch Core

Authored the definitive technical guide on GPU architecture, memory hierarchy, arithmetic intensity, and operator fusion, read by hundreds of thousands of AI infrastructure practitioners worldwide.

Scope & limitations

High-level theoretical roofline models do not account for tail-latency jitter from asynchronous PCIe bus transfers.

Context: NVIDIA Ampere/Hopper architectures, SRAM bandwidth, Tensor Cores, roofline analysis.

View mission
implementationChecked Sep 22, 2026

MCUNet: Tiny Deep Learning on IoT Devices with TinyEngine

Dr. Ji Lin · NVIDIA / MIT

Engineered TinyEngine, a specialized neural network runtime that co-designs model architecture (TinyNAS) and memory scheduling to run deep learning vision and speech models on microcontrollers with only 256KB SRAM.

Scope & limitations

Static memory layout requires exact pre-compilation; dynamic tensor shapes are not supported on microcontrollers.

Context: ARM Cortex-M7 microcontrollers, integer arithmetic, bare-metal C runtime.

View mission
explanationChecked Sep 22, 2026

Medical AI Governance and Patient Privacy Compliance Protocols

Dr. Harvey Castro · Healthcare AI Advisory

Created compliance frameworks for hospital CIOs to evaluate de-identification pipelines, zero data retention BAA contracts, and clinical liability safeguards.

Scope & limitations

EHR vendor legacy APIs (Epic, Cerner) impose strict integration and certification barriers.

Context: HIPAA compliance, BAA agreements, de-identification algorithms, EHR integration.

View mission
implementationChecked Sep 22, 2026

mistral-src: Reference Implementation and MoE Inference Engine

Timothée Lacroix · Mistral AI

Built the official open-source reference codebase for Mistral 7B and Mixtral 8x7B, implementing sliding window attention (SWA), dynamic chunking, and Megatron-style tensor parallel execution for multi-GPU serving.

Scope & limitations

Sliding window attention truncates attention cache for earlier tokens outside the local window unless global tokens are designated.

Context: Sliding window attention (W=4096), grouped-query attention (GQA), PyTorch.

View mission
researchChecked Sep 22, 2026

Mixtral of Experts: Sparse Mixture of Experts with Open Weights

Guillaume Lample · Mistral AI

Co-authored the paper introducing Mixtral 8x7B, a decoder-only model where each layer is composed of 8 feedforward blocks with a router network that selects 2 experts per token, outperforming Llama 2 70B while maintaining the inference speed of a 12B model.

Scope & limitations

MoE models require substantial VRAM for weight storage despite sparse compute activations during generation.

Context: Sparse MoE, 47B total parameters, 13B active per token, 32k context with RoPE.

View mission
implementationChecked Sep 22, 2026

MLX: An Efficient Machine Learning Framework for Apple Silicon

Dr. Awni Hannun · Apple Machine Learning Research

Conceived, engineered, and open-sourced MLX, a NumPy-like array framework with composable function transformations (grad, vmap) that exploits Apple Silicon's unified memory architecture for zero-copy CPU/GPU execution.

Scope & limitations

Exclusively optimized for Apple Silicon hardware; non-macOS platforms are not natively accelerated.

Context: Apple Silicon Metal, unified memory up to 192GB, C++ core with Python bindings.

View mission
implementationChecked Sep 22, 2026

MPT-7B and MPT-30B: Commercial-Grade Open-Source Foundation Models

Dr. Jonathan Frankle · Databricks Mosaic AI / Harvard

Led the engineering and open-weights release of MPT models trained with FlashAttention, ALiBi positional embeddings, and fast streaming data loaders, providing reproducible blueprints for sub-$200k foundation model pre-training.

Scope & limitations

ALiBi positional embeddings exhibit slight performance degradation compared to RoPE when scaled past 128k context windows.

Context: MosaicML Composer, ALiBi 65k context length, 1T token pre-training.

View mission
implementationChecked Sep 22, 2026

Multi-Agent Automation Frameworks and Swarm Blueprints

David Ondrej · Prompt Engineering / Ondrej AI

Maintains widely used open-source developer blueprints for building voice-driven desktop agents, automated research teams, and local rag pipelines.

Scope & limitations

Desktop UI automation relies on OS-level accessibility permissions and screen coordinates.

Context: Python, Streamlit, ChromaDB, Whisper API, Ollama.

View mission
Showing 241–256 of 567 proofs of work