"Writing evals is going to become a core skill for product managers. It is such a critical part of making a good product with AI." — Kevin Weil, CPO@OpenAI

Traditional software was designed to guide users through a single "happy path," but AI products behave differently. Each interaction produces a wide distribution of personalized outcomes for every user. This means the PM craft is fundamentally changing — all the way from writing PRDs to analytics. The AI PM must adapt to this new reality to ship high quality AI products that customers love.

This course is a practical introduction to AI Evaluation for product managers. We will walk through the product management process for AI agents and give you a step-by-step playbook for building an eval-driven roadmap, including AI PRDs, golden datasets, eval rubrics, trace analysis, and automated evaluators. We will use examples, case studies, demos, and DIY exercises to help demonstrate what this process looks like in real life. By the end of the course, our goal is to help you grow into a full stack AI PM who can lead the development of high-quality AI products.