AI research and projects
The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.
A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗
AutoResearch: Autonomous AI Research Agent Conducting Iterative ML Experiments
Andrej Karpathy · Eureka Labs / ex-Tesla, OpenAIAn autonomous multi-step research agent framework designed to conduct end-to-end machine learning experimentation without human intervention. The agent autonomously formulates algorithmic hypotheses, modifies PyTorch model architectures via AST rewriting, schedules multi-GPU training jobs, parses validation loss curves, and generates structured empirical research reports and pull requests.
Requires sandboxed execution environments to prevent arbitrary code execution risks during dynamic model definition; experiments can diverge if loss curves suffer from exploding gradients without bounded learning rate search spaces.
Context: PyTorch 2.4, LLM function calling, Automated Evaluation Loop, Python AST manipulation, TensorBoard metric parsing.
View missionllm.c: Large Language Model Training in Pure C/CUDA Without Dependencies
Andrej Karpathy · Eureka Labs / ex-Tesla, OpenAIA zero-dependency, pure C and CUDA implementation of GPT-2 and modern transformer pre-training. Eliminates the 245MB+ PyTorch/Python runtime overhead by directly implementing forward passes, backward passes, AdamW optimizers, and fused CUDA kernels in a single clean compilation unit, achieving near-parity with PyTorch cuDNN speed.
Lower level of architectural expressiveness compared to dynamic Python autograd graphs; adding novel attention variations requires manual mathematical derivation and custom CUDA kernel writing.
Context: Pure C99, CUDA 12, FP32 and BF16 mixed-precision kernels, fused attention, multi-GPU MPI execution.
View missionmicrograd: Minimal Scalar-Valued Autograd Engine with Dynamic DAG Visualization
Andrej Karpathy · Eureka Labs / ex-Tesla, OpenAIA foundational, pedagogical autograd engine implementing reverse-mode autodiff over dynamically constructed Directed Acyclic Graphs (DAGs) in approximately 100 lines of pure Python, alongside a multi-layer perceptron (MLP) library that replicates PyTorch's core tensor mechanics from first principles.
Operates on individual scalar values rather than vectorized multidimensional tensors, making it unsuitable for training production models beyond small educational datasets.
Context: Pure Python 3.10+, scalar-valued autodiff DAG, Graphviz topological computation graph visualization.
View missionAutomated Code Review and PR Triage AI Assistant
Michael Ballard · SlashdevBuilt a GitHub Action and webhook bot that analyzes incoming pull requests, identifies security vulnerabilities and formatting anti-patterns, and generates unit test coverage proposals automatically.
Large pull requests exceeding 2,000 lines require chunking across logical Git commits to fit within prompt context ceilings.
Context: Claude 3.5 Sonnet, GitHub Octokit API, AST tree-sitter parsers.
View missionAutomated Content Repurposing Engine for Long-Form Media
Paul Macko · Deplatformable / Macko MediaBuilt an end-to-end automated workflow that ingests audio podcasts, generates precise Whisper transcripts, identifies viral hooks using LLM evaluation, and renders styled short-form video clips automatically.
Multi-speaker audio with heavy background music can lead to transcript misalignment without voice separation preprocessing.
Context: Whisper API, Claude 3.5 Sonnet, FFmpeg command automation, Python script runner.
View missionHigh-Speed Optical Character Recognition and Table Extraction Engine
Daniil Komov · SentiSight.ai / NeurotechnologyDeveloped an end-to-end OCR and table structure recognition engine capable of processing warped or skewed scans at under 100ms per page with 98.7% character accuracy across Latin and Cyrillic scripts.
Heavily degraded handwritten documents require specialized handwriting transformer (HTR) checkpoints.
Context: Convolutional recurrent neural network (CRNN) with CTC loss, line segmentation heuristics.
View missionSelf-Healing Kubernetes CI/CD Pipeline with AI Log Anomaly Detection
Michael Ballard · SlashdevEngineered an automated observability pipeline that ingests Kubernetes container logs, flags runtime memory spikes or deadlocks, and automatically recommends or executes configuration rollbacks.
High false-alarm rates during deliberate rolling updates require suppression windows during planned maintenance deployments.
Context: Prometheus, Grafana Loki, Mistral-7B-Instruct fine-tune for root cause analysis.
View missionThe Deplatformable Guide to Local LLMs and Self-Hosted Data Sovereignty
Paul Macko · Deplatformable / Macko MediaPublished an authoritative guide detailing the hardware specifications, quantization trade-offs (GGUF, EXL2), and network sandboxing needed to run private foundation models offline on consumer Apple Silicon and NVIDIA workstations.
Consumer hardware memory limits restrict inference speeds when running 70B parameter models without substantial unified RAM.
Context: Ollama, LM Studio, llama.cpp, Llama 3 8B/70B quantized models.
View missionWeb-Based Deep Learning Model Training and Annotation Platform
Daniil Komov · SentiSight.ai / NeurotechnologyArchitected a cloud-based platform enabling non-technical users to annotate images with bounding boxes, polygons, and keypoints, and train custom classification, object detection, and OCR models with one click.
Training speed depends on GPU cluster queue availability during peak multi-user training workloads.
Context: PyTorch, ONNX Runtime, WebSockets for live training metrics, automated augmentations.
View missionA Path Towards Autonomous Machine Intelligence (JEPA Architecture Framework)
Dr. Yann LeCun · Meta / New York UniversityPublished the comprehensive theoretical blueprint proposing Joint Embedding Predictive Architecture (JEPA) as an alternative to generative pixel-level or token-level prediction, learning representations in abstract embedding space to enable hierarchical planning.
Action-conditional latent dynamics models remain active research topics; translating abstract latent representations back into fine-grained execution paths requires auxiliary decoding heads.
Context: Self-supervised non-contrastive energy-based models; architectural basis for I-JEPA (vision) and V-JEPA (video).
View missionAI Snake Oil: What Computers Can, Can't, and Shouldn't Do
Dr. Sayash Kapoor · Princeton UniversityDeveloped the analytical distinction between perception AI (generative synthesis, speech recognition) and predictive AI (predicting human recidivism, job success), providing rigorous frameworks for evaluating claims of AI accuracy in business and governance.
Practical applications in enterprise require navigating existing regulatory guidelines that may lag behind empirical machine learning consensus.
Context: Statistical risk assessment, cross-entropy evaluation, algorithmic bias metrics in high-stakes human systems.
View missionAttention Is All You Need: The Seminal Transformer Architecture Paper
Aidan Gomez · CohereCo-authored the groundbreaking NeurIPS 2017 research paper introducing the Transformer, an architecture dispensing entirely with recurrence and convolutions in favor of multi-head self-attention mechanisms, founding modern LLM technology.
Original architecture scaled quadratically O(N^2) with sequence length, prompting subsequent research into grouped-query attention, sparse attention, and linear recurrence hybrids.
Context: 8-head multi-head self-attention, sinusoidal positional encodings, trained on 8 NVIDIA P100 GPUs.
View missionAuditing Multimodal Generative Systems for Representational and Allocative Harm
Dr. Margaret Mitchell · Hugging FaceFormulated structured auditing protocols for text-to-image and multimodal foundation models, measuring bias propagation across CLIP embeddings, latent diffusion prompt expansions, and synthetic face generation.
Balancing automated quantitative metric auditing with contextual qualitative evaluation requires domain-specific sociological input.
Context: CLIP, Stable Diffusion, demographic diversity benchmarking over fine-grained prompt variations.
View missionBottleneck Transformers for Visual Recognition (BoTNet)
Aravind Srinivas · Perplexity AICo-authored BoTNet, an architecture that replaces the final spatial convolutions in ResNet with Multi-Head Self-Attention, achieving 84.7% Top-1 accuracy on ImageNet while substantially improving instance segmentation and object detection on COCO.
Quadratic memory complexity in early spatial layers restricts attention blocks to deeper, downsampled feature maps in the network.
Context: ResNet backbone with Multi-Head Self-Attention bottleneck blocks, TPU v3 accelerators, 2D relative position encodings.
View missionChain-of-Thought Prompting Elicits Reasoning in Large Language Models
Dr. Jason Wei · OpenAIAuthored the seminal NeurIPS 2022 research paper demonstrating that generating intermediate reasoning steps dramatically boosts LLM performance on arithmetic, symbolic, and multi-hop reasoning tasks.
Chain-of-thought gains exhibit emergence only at significant model scale (~50B+ parameters); smaller models can generate superficially coherent rationales that lead to incorrect arithmetic conclusions.
Context: PaLM 540B, GPT-3 175B, GSM8K, SVAMP, BIG-bench reasoning tasks.
View missionCohere Command R+ Enterprise Agentic Retrieval & Tool Calling Engine
Aidan Gomez · CohereDirected the development of Command R+, an open-weights 104B parameter model optimized specifically for multi-step agentic workflows, multi-lingual enterprise Retrieval-Augmented Generation (RAG), and cited tool calling across complex relational schemas.
Heavy memory footprint requiring 4x NVIDIA A100 (80GB) or quantized 8-bit / 4-bit deployments in production environments.
Context: 104B parameters, 128k context window, native citation grounding, multi-turn tool use API schema.
View mission