Woosuk Kwon
vLLM / UC Berkeley
Co-creator and project lead of vLLM, and researcher at UC Berkeley SkyLab. Invented PagedAttention, a virtual memory paging algorithm for LLM key-value caches that eliminated memory fragmentation and increased LLM serving throughput by 2-4x across the AI industry.
Areas of focus
Professional niches
Proof of Work
vLLM: High-Throughput and Memory-Efficient LLM Serving Engine
Engineered and open-sourced vLLM, an enterprise serving system providing continuous batching, chunked prefill, speculative decoding, and tensor parallelism, serving billions of production inference tokens daily worldwide.
Requires dedicated GPU clusters and NVLink interconnects for multi-node tensor-parallel deployments.
Context: CUDA/ROCm kernels, Triton backends, Ray distributed orchestration, vLLM OpenAI-compatible server.
View missionEfficient Memory Management for Large Language Model Serving with PagedAttention
Authored the seminal SOSP 2023 paper introducing PagedAttention, inspired by operating system virtual memory paging, allowing non-contiguous KV-cache storage and near-zero memory waste during high-concurrency LLM inference.
Paged cache lookup introduces minor pointer indirection overhead in memory-bandwidth-bound token generation loops.
Context: Evaluated across LLaMA, OPT, and Falcon architectures with continuous batching and multi-query attention.
View mission