AI research and projects
The research, projects, and explanations behind the people. Read the source, inspect verified code and models, and find a collaborator.
A source check confirms that a source supports the summary. It is not an independent replication or endorsement. Our methodology ↗
Holistic Evaluation of Language Models (HELM)
Dr. Percy Liang · Stanford UniversityA multi-dimensional benchmark evaluating foundational LLMs across 42 real-world scenarios and 7 evaluation metrics to eliminate selective benchmark reporting.
Continuous maintenance required as frontier model providers iterate closed alignment weights.
Context: Empirical evaluation suite covering all major commercial and open foundation models.
View missionBlueprint for an AI Bill of Rights: Making Automated Systems Work for the American People
Dr. Alondra Nelson · Institute for Advanced Study / ex-White House OSTPFoundational national policy framework establishing five core principles for AI governance: Safe and Effective Systems, Algorithmic Discrimination Protections, Data Privacy, Notice and Explanation, and Human Alternatives.
Serves as federal policy and guidance; statutory enforcement requires agency-specific rulemaking or congressional legislation.
Context: Federal technology policy, socio-technical evaluation, and automated decision systems.
View missionPrompt Injection and Jailbreaking: Threat Modeling Autonomous LLM Tool-Call Security
Simon Willison · Datasette / IndependentComprehensive research series documenting indirect prompt injection attack vectors, markdown image exfiltration vulnerabilities, and defense-in-depth isolation strategies for autonomous agents.
Prompt filtering heuristics fail against novel encoded or multi-turn attacks; robust security requires architectural data/instruction boundary isolation.
Context: LLM tool-call protocols, SQLite, Datasette, Python, and web security.
View missionTargeted Real-Time Early Warning System for Sepsis in Clinical Inpatient Settings
Dr. Suchi Saria · Johns Hopkins University / Bayesian HealthMulti-center prospective clinical study evaluating a real-time predictive ML system integrated with EHRs, demonstrating an 18.2% reduction in sepsis patient mortality with high provider adoption.
Requires real-time streaming integration with clinical EHR databases (Epic/Cerner) and hospital IT network compliance.
Context: Causal inference models, electronic health record data pipelines, and HL7/FHIR integration.
View missionFlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Tri Dao · Together AI / Carnegie Mellon UniversitySeminal hardware-aware algorithm optimizing GPU SRAM and HBM memory reads/writes to compute exact attention in sub-quadratic memory, delivering 2-4x speedups across major foundation models.
Requires GPU compute capability >= 8.0 (NVIDIA Ampere, Ada, or Hopper) and compiled CUDA/Triton kernels.
Context: CUDA C++, Triton, PyTorch C++ extensions, and modern GPU architectures.
View missionMeasuring Massive Multitask Language Understanding (MMLU)
Dr. Dan Hendrycks · Center for AI SafetyThe industry-standard academic benchmark measuring world knowledge and problem-solving across 57 subjects ranging from elementary mathematics to professional law and medicine, authored by Dr. Dan Hendrycks.
Multiple-choice evaluation format does not directly measure multi-step agent reasoning, tool usage, or long-form generation coherence.
Context: Multiple-choice evaluation suite across 57 distinct disciplines.
View missionMozilla Foundation / UC Berkeley: Enterprise AI Risk Management & Governance Framework
Inioluwa Deborah Raji · Mozilla Foundation / UC BerkeleyA structured governance and compliance methodology authored by Inioluwa Deborah Raji at Mozilla Foundation / UC Berkeley, establishing organizational guardrails, vendor evaluation criteria, and model validation standards for enterprise adoption.
Strategic governance advisory framework; regulatory compliance requirements must be validated against regional legal statutes.
Context: Computer vision benchmarks, statistical auditing methodologies, and demographic parity metrics.
View mission