PUBLISHED AI RESEARCH AND PROJECTS

AI research and projects

The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.

Clear all filters

A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗

Showing 33–48 of 149 proofs of work · Kind: research · Newest first
Page 3 of 10
researchChecked Sep 23, 2026

Scaling Laws for Neural Language Models

Dario Amodei · Anthropic

Discovered empirical power-law relationships between transformer performance (cross-entropy loss) and model parameter count, dataset size, and compute budget spanning over six orders of magnitude, providing the blueprint for modern frontier foundation model investment.

Scope & limitations

Scaling power laws hold primarily for pretraining loss; downstream task sample efficiency and multi-step reasoning require structured inference-time compute scaling.

Context: Autoregressive language models scaled from 10^3 to 10^9 non-embedding parameters across WebText.

View mission
researchChecked Sep 23, 2026

Sequence to Sequence Learning with Neural Networks

Ilya Sutskever · Safe Superintelligence Inc. (SSI)

Pioneered the multi-layered Long Short-Term Memory (LSTM) encoder-decoder architecture mapping arbitrary sequence lengths to variable output sequences. Revolutionized machine translation, conversational AI, and established the direct conceptual architecture upon which transformers and autoregressive LLMs were built.

Scope & limitations

Recurrent execution bottlenecked parallelization across large cluster topologies before self-attention mechanisms emerged.

Context: Deep LSTM networks, WMT14 English-to-French benchmark, 160,000 input vocabulary.

View mission
researchChecked Sep 23, 2026

Show and Tell: A Neural Image Caption Generator

Oriol Vinyals · Google DeepMind

Pioneered the integration of deep convolutional neural networks (CNNs) for vision encoding with recurrent language models (RNNs) for text generation, producing the first end-to-end neural image captioning pipeline.

Scope & limitations

Susceptible to hallucinating objects based on strong linguistic prior frequencies rather than verified visual presence.

Context: GoogLeNet CNN feature extractor combined with LSTM language decoder on MSCOCO dataset.

View mission
researchChecked Sep 23, 2026

Sparks of Artificial General Intelligence: Early experiments with GPT-4

Sebastien Bubeck · Microsoft Research

Authored the 154-page seminal investigation demonstrating that early non-multimodal GPT-4 exhibited emergent general intelligence across mathematics, coding, law, vision-via-code, psychology, and medicine without domain-specific fine-tuning.

Scope & limitations

Highlighted critical remaining flaws including lack of long-term planning, working memory degradation, and blind confidence in false premises.

Context: Unconstrained early GPT-4 base model checkpoint evaluated across novel out-of-distribution reasoning probes.

View mission
researchChecked Sep 23, 2026

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Jeff Dean · Google / Google DeepMind

Co-architected TensorFlow, the open-source computation graph framework that unified distributed deep learning execution across mobile devices, CPU servers, GPU clusters, and TPU pods, catalyzing the deep learning revolution.

Scope & limitations

Static computational graph compilation initially hindered dynamic Pythonic debugging compared to eager execution paradigms.

Context: Heterogeneous distributed computation graphs supporting thousands of distributed workers.

View mission
researchChecked Sep 23, 2026

Textbooks Are All You Need (The Phi Model Series)

Sebastien Bubeck · Microsoft Research

Demonstrated that a 1.3-billion parameter transformer trained on 6 billion tokens of curated synthetic 'textbook quality' data achieved 50.6% on HumanEval, dramatically outperforming models 10x its size trained on hundreds of billions of uncurated web tokens.

Scope & limitations

Synthetic data diversity must be aggressively managed to prevent model collapse and repetition loops on out-of-distribution prompts.

Context: Phi-1 and Phi-2 autoregressive models trained on synthetic Python exercises and educational textbooks curated by GPT-3.5.

View mission
researchChecked Sep 23, 2026

The Representation of Truth & World Models in Large Language Models

Max Tegmark · MIT / Future of Life Institute

Discovered that large language models build linearly readable internal representations of truth and physical space/time, demonstrating that models do not merely emulate shallow stochastic parrots but construct coherent world models.

Scope & limitations

Probes capture model belief states which can diverge from actual external physical reality.

Context: Llama-2 and Pythia checkpoints probed via principal component analysis and logistic probes across factual statements.

View mission
researchChecked Sep 22, 2026

A Foundation Model for Clinician-Centered Drug Repurposing with TxGNN

Dr. Kexin Huang · Stanford University / Harvard University

First author of the Nature Medicine 2024 paper introducing TxGNN, a geometric foundation model trained on a medical knowledge graph of 17,080 diseases to identify drug repurposing opportunities for rare diseases with clinician-interpretable rationales.

Scope & limitations

Predictions require in vitro wet-lab assay validation and clinical trials before therapeutic efficacy can be established.

Context: Graph Neural Networks, 17k disease knowledge graph, clinician explanatory subgraphs.

View mission
researchChecked Sep 22, 2026

Aggregated Residual Transformations for Deep Neural Networks (ResNeXt)

Dr. Saining Xie · New York University (NYU)

Co-authored the seminal CVPR 2017 paper introducing ResNeXt, which introduced 'cardinality' (the size of the set of transformations) as a concrete, essential dimension that outclasses depth and width for computer vision accuracy.

Scope & limitations

Grouped convolution memory access patterns can yield lower tensor-core utilization compared to standard dense GEMMs.

Context: Grouped convolutions, split-transform-merge topology, ImageNet-1k classification.

View mission
researchChecked Sep 22, 2026

AWQ: Activation-aware Weight Quantization for On-Device LLM Compression

Dr. Ji Lin · NVIDIA / MIT

Authored the MLSys 2024 Best Paper introducing AWQ, demonstrating that protecting the top 1% salient weight channels based on activation magnitudes avoids backpropagation retraining and achieves lossless 4-bit inference.

Scope & limitations

Extreme 2-bit quantization still requires second-order error compensation beyond magnitude scaling.

Context: W4A16 GEMM kernels, LLaMA, Mistral, CodeLlama, NVIDIA TensorRT-LLM and vLLM runtimes.

View mission
researchChecked Sep 22, 2026

Benchmarking Vision-Language Models on Live Web Interaction

Gregor Zunic · Browser-Use

Conducted empirical benchmarks assessing vision versus accessibility-tree DOM representation for autonomous agents, proving multimodal screenshot parsing reduces failure rates by 35% on checkout flows.

Scope & limitations

High resolution screenshot ingestion increases token consumption significantly per navigation step.

Context: Claude 3.5 Sonnet vision, WebVoyager benchmark dataset, automated browser sessions.

View mission
researchChecked Sep 22, 2026

Benchmarking Web Extraction Quality for Large Language Models

Minh Do · Crawl4AI

Conducted empirical benchmarks demonstrating a 6x speedup and 70% reduction in context token waste by stripping non-semantic DOM elements prior to RAG embedding.

Scope & limitations

Complex nested web tables with multiple header levels require custom parsing logic.

Context: DOM pruning algorithms, cosine similarity RAG retrieval, token economics.

View mission
researchChecked Sep 22, 2026

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Dr. Thomas Wolf · Hugging Face

Initiated and directed the BigScience open scientific collaboration of 1,000+ researchers, pre-training BLOOM on the Jean Zay supercomputer across 46 natural languages and 13 programming languages.

Scope & limitations

Massive parameter size requires distributed multi-node hardware (8x A100 80GB minimum) for unquantized inference.

Context: 176B parameters, Megatron-DeepSpeed 3D parallelism, ALiBi positional embeddings.

View mission
researchChecked Sep 22, 2026

Colorful Image Colorization with Deep Convolutional Neural Networks

Dr. Alexei Efros · UC Berkeley

Pioneered automated black-and-white image colorization by formulating the task as multinomial classification over quantized Lab color space with class rebalancing, tricking human observers into believing synthesized colors are real.

Scope & limitations

Ambiguous objects (e.g. clothing, cars) can display desaturated or sepia tints if the model averages multimodal color distributions.

Context: CIE Lab color space, class-rebalanced cross-entropy loss, 1.3M ImageNet images.

View mission
researchChecked Sep 22, 2026

Connectionist Temporal Classification: Labelling Unsegmented Sequence Data

Dr. Alex Graves · NNAISENSE / former Google DeepMind

Authored the monumental ICML 2006 paper introducing CTC loss, allowing RNNs to be trained directly on unsegmented, unaligned sequence pairs by integrating over all possible alignments with a blank token, with over 15,000 citations.

Scope & limitations

Assumes conditional independence between label outputs given the input sequence, necessitating external language models for grammar smoothing.

Context: Dynamic programming forward-backward algorithm, RNN sequence alignment.

View mission
researchChecked Sep 22, 2026

Contrastive Multiview Coding and Self-Supervised Representation Learning

Dr. Phillip Isola · MIT

Introduced Contrastive Multiview Coding (CMC), showing that maximizing mutual information across multiple views or sensory modalities learns invariant visual representations competitive with full ImageNet supervision.

Scope & limitations

Requires view-generation or multiple aligned cameras/modalities during pre-training to compute contrastive pairs.

Context: Multi-sensory embedding spaces, contrastive InfoNCE loss.

View mission
Showing 33–48 of 149 proofs of work