Ali Ghodsi
Co-Founder & CEO, Databricks | Adjunct Professor, UC Berkeley | Co-Creator of Apache Spark
Co-founder and CEO of Databricks, the multi-billion-dollar enterprise data and AI company. Adjunct Professor at UC Berkeley EECS. Co-creator of Apache Spark, Delta Lake, MLflow, and the Data Intelligence Platform. Spearheaded Databricks' pivotal acquisitions of MosaicML and Lilac AI, pioneering compound AI systems and enterprise foundation model training that combine proprietary tabular enterprise data with generative LLMs.
Areas of focus
Professional niches
Proof of Work
Databricks Mosaic AI & Lakehouse Architecture for Foundation Models
Led the engineering integration of MosaicML training and fine-tuning pipelines directly into the Databricks Lakehouse, enabling organizations to pretrain and align custom open-weight models securely on internal enterprise data lakes.
On-premise and private cloud cluster provisioning constraints can bottleneck distributed tensor parallelism across non-homogeneous GPU topologies.
Context: DBRX, MPT-7B/30B, MosaicML Composer, Delta Lake, Vector Search.
View missionThe Shift from Monolithic Models to Compound AI Systems
Co-authored the influential technical treatise establishing why state-of-the-art AI applications are moving from single monolithic LLMs to compound AI systems integrating multi-step reasoning, vector databases, code execution, and external verification tools.
Multi-component compound architectures multiply system failure modes, requiring end-to-end tracing and specialized evaluation harnesses.
Context: Compound AI architectures, RAG pipelines, LLM routing, Apache Spark, MLflow.
View mission