AI Engineering by Chip Huyen — Review

Chip Huyen's Designing Machine Learning Systems became the default text for ML platform engineering after its 2022 release. Her follow-up, AI Engineering, tackles the same audience (software engineers working on AI systems) but pivoted to the foundation-model era: LLMs, RAG, prompt engineering, and agentic systems.

What The Book Covers

• Foundation model fundamentals (why they work, what they can't do)
• Model selection and evaluation (which LLM for which task)
• Prompt engineering (beyond "be specific" — structured prompting, chain-of-thought, few-shot)
• RAG architecture (retrieval, chunking strategies, vector DBs, rerankers)
• Fine-tuning (when it's worth it, how to do it without breaking the base)
• Agentic systems (tool use, multi-step reasoning, safety boundaries)
• Evaluation (eval frameworks, offline and online eval design)
• Production concerns (latency, cost, caching, monitoring)

Strongest Chapters

The RAG chapter. Most RAG writing online is either "here's a LangChain demo" or academic. Huyen's treatment covers the actual production decisions: chunk sizes under what conditions, hybrid search vs dense-only, how to handle freshness, reranking models and their trade-offs.

The evaluation chapter. LLM app evaluation is the biggest gap in most teams' workflows. Huyen walks through eval pipeline design, LLM-as-judge, offline benchmarks, and online A/B testing with statistical rigor.

The agentic systems chapter. Covers multi-agent architectures (which patterns work, which don't), tool-calling reliability, and safety boundaries. Rare rigor in a space dominated by tutorials.

What's Missing

Specific model benchmarks. The book is deliberately model-agnostic. Given how fast the leaderboard changes (Claude 3.7 → 4.7, GPT-4 → GPT-5), this is the right choice — but you'll still need current benchmarks when making model selections.

Depth on specific tools. LangChain, LlamaIndex, DSPy, Haystack get mentioned but not deep-dived. Huyen's focus is architecture, not tool-specific.

Who Should Read

Every software engineer building LLM-backed applications in production. Tech leads and architects making build-vs-buy decisions on AI systems. Product managers who need to understand the architecture of what their team is shipping.

Who Should Skip

Pure ML researchers — this is an engineering book, not a paper-replication manual. Weekend hackers on their first LLM project — start with simpler material first.

Verdict

The right book at the right time for engineering teams scaling from "LLM prototype" to "LLM in production." Read cover-to-cover, keep as reference throughout the year.