Dr. Ji Lin
Senior Research Scientist
Senior Research Scientist at NVIDIA and former MIT PhD researcher. First author of AWQ (Activation-aware Weight Quantization) and MCUNet, pioneering hardware-friendly low-bit compression algorithms that enable running frontier models on edge accelerators and consumer GPUs.
Areas of focus
Professional niches
Proof of Work
AWQ: Activation-aware Weight Quantization for On-Device LLM Compression
Authored the MLSys 2024 Best Paper introducing AWQ, demonstrating that protecting the top 1% salient weight channels based on activation magnitudes avoids backpropagation retraining and achieves lossless 4-bit inference.
Extreme 2-bit quantization still requires second-order error compensation beyond magnitude scaling.
Context: W4A16 GEMM kernels, LLaMA, Mistral, CodeLlama, NVIDIA TensorRT-LLM and vLLM runtimes.
View missionMCUNet: Tiny Deep Learning on IoT Devices with TinyEngine
Engineered TinyEngine, a specialized neural network runtime that co-designs model architecture (TinyNAS) and memory scheduling to run deep learning vision and speech models on microcontrollers with only 256KB SRAM.
Static memory layout requires exact pre-compilation; dynamic tensor shapes are not supported on microcontrollers.
Context: ARM Cortex-M7 microcontrollers, integer arithmetic, bare-metal C runtime.
View mission