Design experiments that survive real scrutiny.
Go from causal question to production experiment with hypotheses, metrics, sample size, randomization, diagnostics, CUPED, sequential testing, and bandits.
What you will leave with
You can write an experiment plan, calculate its requirements, diagnose broken assignment, analyze results, and communicate a defensible decision.
What you will be able to do
The curriculum is organized around these 4 practical outcomes.
Turn a product question into a falsifiable hypothesis
Define primary, secondary, and guardrail metrics
Plan sample size and detect randomization failures
Apply variance reduction and sequential methods responsibly
Course map
Curriculum
Open a module to inspect every section before you start. Your progress follows you through the course.
Correlation vs causation, The counterfactual, Random assignment, and more.
Hypothesis writing, OEC primary metric, Secondary metrics, and more.
Null hypothesis, p-values, Confidence intervals, and more.
Statistical power, Effect size (MDE), Sample size formulas, and more.
Random assignment, Deterministic hashing, Sticky vs session-level, and more.
Peeking problem, Multiple testing, Simpson's paradox, and more.
Variance is the enemy, CUPED, Stratification, and more.
Multi-armed bandits, Switchback experiments, Bayesian A/B, and more.
Who this course is for
Start with the background you have. The prerequisite notes above tell you exactly what is assumed.
Product and data analysts
Data scientists responsible for experimentation
PMs and engineers who review test results
Continue learning
Each course stands on its own. Pick the one closest to the next task you need to handle.
Build statistical intuition before formulas.
Reason clearly when outcomes are uncertain.
Train models the way working data scientists do.
Start the course
Module 1 introduces the language and example used throughout the rest of the course.
