Part I
Inside the model
What actually happens to your text inside a transformer.
37 chapters · live, interactive diagrams
Watch an LLM serve a request,
token by token.
A visual, hands-on guide to LLM inference and serving. It follows a prompt through tokenization, attention, KV caches and GPU kernels, then through batching, schedulers, parallelism and the production stack that streams the answer back.
The full lifecycle
One request, end to end
The animation runs in slow motion. Click any stage to open the chapter that explains it.
Phase—
TTFT (slow-mo)—
Tokens out0
KV cache0 tokens
Learn by doing
Three ways in
Experiment first and read later, or the other way round.
Inference LabTurn every knob at oncePick a model, GPU and workload. Memory, step time, the roofline, the throughput–latency curve and cost all update together, with a plain-English note on what is limiting you.Open the Lab →Concept MapSee how the ideas connectAll 37 chapters and the links between them. Click any topic for the shortest reading path to it.Explore the map →GlossaryNever get stuck on a termTTFT, GQA, MIG, goodput… Short definitions, also shown on hover inside every chapter.Browse terms →
Curriculum
Eight parts, from a single matmul to a GPU fleet
Read it in order or jump around. Every chapter has live diagrams and ends with interview-style questions.
Part II
One request, one GPU
Where time and memory go when a single prompt runs.
Part III
Many requests, one engine
How serving engines juggle hundreds of users at once.
Part IV
Faster & cheaper
Tricks that buy speed, memory and features.
Part V
Scaling out
When one GPU is not enough.
Part VI
Production
Measuring, routing, scaling and paying for it.
Part VII
Beyond chat
Reasoning, agents, images and embeddings stress the engine differently.
Part VIII
Running on Kubernetes
Getting GPUs, weights and visibility into a cluster.
Visual language
Learn the colours once
Every diagram on the site uses the same colour for the same concept, so you can read a new chart at a glance.
PrefillProcessing the prompt: one wide, compute-bound pass
DecodeGenerating one token per step: memory-bound
KV cacheKeys and values kept for every past token
WeightsModel parameters streamed from HBM
TokensToken IDs, vocab, sampling
ComputeFLOPs on tensor cores
MemoryIdle, queued, overhead
CommunicationNVLink, all-reduce, KV transfer
WasteFragmentation, stalls, recompute
Live, not staticPlay, pause, step and scrub every simulation, or change the inputs and watch the result update.
Real numbersCalculators use real model configs (Llama, Qwen, DeepSeek, Mixtral) and real GPU specs (A100 → B200).
Interview-readyEach chapter closes with core, mid and senior-level questions you can try before revealing the answer.
Keyboard-first⌘K or / to search, [ and ] for the previous and next chapter.