Understand what happens between tokens and text.
Build a connected mental model of tokenization, embeddings, attention, transformer blocks, training, decoding, post-training, and interpretability.
What you will leave with
You can trace an LLM request through the major mechanisms, explain the tradeoffs, and reason about failures without relying on slogans.
What you will be able to do
The curriculum is organized around these 4 practical outcomes.
Explain how text becomes tokens and vectors
Trace information through attention and transformer blocks
Connect training objectives to model behavior
Reason about decoding, post-training, and interpretability
Course map
Curriculum
Open a module to inspect every section before you start. Your progress follows you through the course.
BPE, Vocabulary, Glitch tokens, and more.
Word embeddings, Sinusoidal positions, RoPE, and more.
Q/K/V, Scaled dot-product, Multi-head, and more.
LayerNorm vs RMSNorm, SwiGLU FFN, Pre-norm, and more.
Next-token loss, AdamW, Scaling laws, and more.
Greedy vs sampling, Temperature, Top-k / top-p / min-p, and more.
SFT, DPO, GRPO / RLVR, and more.
Mechanistic interpretability, Induction heads, Sparse autoencoders, and more.
Who this course is for
Start with the background you have. The prerequisite notes above tell you exactly what is assumed.
Engineers entering LLM application work
Data professionals who need model-level intuition
Technical leaders evaluating LLM claims
Continue learning
Each course stands on its own. Pick the one closest to the next task you need to handle.
Build a working language model from scratch.
Build retrieval systems you can measure.
Build the machinery around an agent loop.
Start the course
Module 1 introduces the language and example used throughout the rest of the course.
