A/B Testing & Experimentation.
From Coin Flip to Production.

Why dashboards lie. Why correlation isn't causation. Why a coin flip is the single most powerful tool in your toolkit. Set up Bean & Brew's free-shipping test.

The #1 mistake junior data scientists make. Falsifiable hypotheses, the OEC, secondary metrics that explain mechanism, and guardrails that protect what matters.

p-values, confidence intervals, Type I & II errors — done the way an experimenter needs them, not the way a textbook teaches. Two-sample tests for proportions and means.

The four knobs every experiment turns: effect size, alpha, power, variance. The math behind 'how long do I run this?' and the trade-offs nobody warns you about.

The engineering side every course skips. Deterministic hashing, sticky assignment, salts and rotations, and the SRM canary that catches 90% of platform bugs.

Peeking, multiple testing, Simpson's paradox, novelty effects, weekend skew. The interview-grade catalogue of how experiments lie — and the corrections that fix them.

CUPED, stratification, and sequential testing — what L5+ data scientists at Meta, Google, Netflix, and Microsoft use to run experiments in half the time.

Bandits, switchback, Bayesian A/B, and quasi-experiments for when randomization is impossible. The interview separator. Capstone: run Bean & Brew's test end-to-end.