Hamel Husain
Founder, Parlance Labs | AI Evaluation Architect & LLM Systems Consultant
Hamel Husain is an AI consultant and founder of Parlance Labs. Previously a Staff Machine Learning Engineer at GitHub, Airbnb, and DataRobot, Hamel teaches engineers how to build, evaluate, and fine-tune production LLM systems. His guides on evaluation-driven development and domain-specific fine-tuning are widely used across top AI startups.
Areas of focus
Professional niches
Proof of Work
Fine-Tuning Open Source LLMs: A Hands-On Guide for Engineers
A comprehensive, practitioner-focused field guide for evaluating and fine-tuning open-source models using Axolotl, LoRA, and curated evaluation benchmarks. Covers data synthesis, loss curve diagnosis, and catastrophic forgetting mitigation.
Focuses on instruction-tuning and task adaptation; pre-training from scratch requires massive compute clusters outside the scope of this guide.
Context: Tested across LLaMA, Mistral, and DeepSeek model families using Axolotl and Hugging Face PEFT.
View missionYour AI Product Needs Evals: Comprehensive Guide to Unit Testing & LLM Evaluation
Comprehensive framework for creating automated evaluation loops, synthetically augmented test sets, and LLM-as-a-judge scoring harnesses that measure production degradation before deployment.
LLM-as-a-judge evaluators can exhibit self-preference bias and positional bias if uncalibrated against human baseline judgments.
Context: Promptfoo, Braintrust, Python, OpenAI, Anthropic, and OpenPipe.
View mission