Start here
LLM by Hand teaches how large language models work in one way: you compute every number yourself. This page says what you need to know first, what you need to have, how a level works, and how to get the most out of it.
What you need to know
Basic Python
Variables, lists, for loops, and functions with def. The code you fill in is short. You don’t need NumPy or PyTorch yet: the course teaches them, NumPy from level 1 and PyTorch in level 9.
School math
Arithmetic and basic algebra are enough. Everything else the course uses (e and ln, the Σ sign, mean and variance, derivatives, the chain rule) is on the math page, each with a small example in numbers. Lessons link to it where it comes up.
Not needed
- A course in linear algebra or calculus
- Machine learning experience
- A GPU or a paid service
What to have: a browser (a phone works, but the code questions are easier on a computer) and pen and paper. Almost everything runs in your browser, the Python questions included. Only a few parts run on your own computer: the bosses that train real models with PyTorch, and a few optional runs. When you get there, the setup guide walks you through it in about ten minutes.
How a level works
Each level answers one question, written under its title. The answer is built from an example small enough to compute on paper: two or three words, vectors with two numbers, three points.
- LabThe picture and the numbers move together. Drag something or change a number, and watch what follows.
- Number / ShapeType a number, or a shape like
(2, 3). A wrong answer gets a hint, and each hint says a little more. - Predict firstPick an answer before you see what happens. Being wrong is fine; what matters is committing first.
- CodeFill in a few lines of Python, run them in the browser, and pass the tests.
- Read onQuestions never block the page. If you are stuck, keep reading and come back. Stars come from the questions you answer.
- I got stuck hereA real question a learner asked at that spot, with its answer. “Go deeper” boxes hold derivations and design reasons. Both are optional.
- RecapOpens when you clear the level: what you can now do, the formulas and shapes to keep, the common mistakes.
Clearing a level gives stars: ★★★ every question solved on your own, ★★☆ some with the answer shown, ★☆☆ cleared with some questions left. The stars are for you.
From level 2 on, each level starts with a two-question warm-up from earlier levels, so you review before you forget. If you already know a level, use “Already know this? Skip it with a short test”: solve a few of its questions on your own and it counts as cleared.
How to learn well
- Compute on paper, then type. The examples are small so that you can. Working one out by hand sticks better than reading it ten times.
- Guess before you look. A “Predict first” question only works if you really commit. The moment a guess turns out wrong is when you learn.
- Read the hint and try again. Keep “Show answer” for when you are truly stuck: a shown answer keeps the level at two stars at most.
- Play with the labs. Before you change a number, guess what will happen, then check.
- Walk the main line in order. Each level builds on the ones before it. Side trips are optional; take them when you are curious.
- Come back. Questions you missed go to Mistakes; redo them a few days later. The warm-ups bring old questions back too.
- Just want to read? Turn on reading mode in settings (⚙): every lab value shows, nothing is recorded, and no stars are given.
The route
The main line has 21 levels in two parts. From level 1 to level 21 you end up writing a GPT that learns two-digit addition.
Foundations · how numbers learn
Theory · what the model computes
Side trips open after a level on the main line, and the sidebar and the map show them next to it: Under the hood (U1–U6) opens up what PyTorch hides; Classic networks (N1–N7) covers the networks before the Transformer, up to the first Transformer of 2017; Diffusion (D1–D3) is another way to generate. The main line never needs them.
Know neural networks already? You can start at level 11 and skip the earlier levels with their short tests.
Your progress
There is no account. Your answers and stars are saved in this browser only. To move to another browser or computer, export a progress file in settings (⚙) and import it there.
Start with level 1 Matrices and shapes