Dr. Sheng Shen
AI Research Scientist
AI Research Scientist at UC Berkeley BAIR lab and co-creator of SGLang and HuggingGPT. Specializes in efficient memory management, structured constrained decoding, and multi-agent LLM systems connecting foundation models with specialized software tools.
Areas of focus
Professional niches
Proof of Work
Efficient Memory Management for Structured Generation in Large Language Models
Formulated algorithms for structured JSON and regex-constrained LLM generation that maintain jump-forward finite-state machines without re-allocating KV caches, accelerating structured extraction by 3x.
Complex nested recursive schemas require state graph compilation before first token generation.
Context: Constrained grammar parsing, Outlines/XGrammar integration, LLaMA-2/3.
View missionHuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Developed HuggingGPT (JARVIS), a pioneering multi-agent framework that uses an LLM controller to parse user intent, plan task dependency graphs, dispatch subtasks to Hugging Face expert models, and synthesize results.
Multi-hop tool routing introduces latency cascades if intermediate domain models fail or return incompatible schemas.
Context: Multimodal coordination across vision, audio, text, and generative models.
View mission