Horace He
AI Compiler Architect
AI Compiler Architect at Thinking Systems and former core team member of PyTorch at Meta AI. Co-creator of PyTorch 2.0 (`torch.compile`), TorchDynamo, and TorchInductor, transforming dynamic Python execution into high-performance fused Triton kernels.
Areas of focus
Professional niches
Proof of Work
PyTorch 2: Dynamic Python Bytecode Compilation and TorchInductor
Co-authored the ACM ASPLOS 2024 paper detailing TorchDynamo and TorchInductor, which intercept Python frame evaluation bytecodes to capture computational graphs safely and generate fused Triton GPU kernels with 0% code modifications.
Dynamic control flow with non-tensor Python data structures forces graph breaks that require partial eager execution fallback.
Context: TorchDynamo, TorchInductor, OpenAI Triton backend, 160+ benchmark models.
View missionMaking Deep Learning Go Brrrr: Performance Engineering and GPU Kernel Optimization
Authored the definitive technical guide on GPU architecture, memory hierarchy, arithmetic intensity, and operator fusion, read by hundreds of thousands of AI infrastructure practitioners worldwide.
High-level theoretical roofline models do not account for tail-latency jitter from asynchronous PCIe bus transfers.
Context: NVIDIA Ampere/Hopper architectures, SRAM bandwidth, Tensor Cores, roofline analysis.
View mission