Skip to content

43 chapters · 550 lessons · about 93 hours

From one neuron to agents

One road in three stages. The level test finds where you join it.

Stage 1 · Beginner

How Machines Learn

From one neuron to a network you train yourself.

No background needed. By the end you can explain how a machine learns from examples, and train a small neural network.

    1. 1A hundred models in your pocket10′
    2. 2Rules, or examples9′
    3. 3Data is just examples9′
    4. 4Finding the pattern9′
    5. 5A guess with a number on it9′
    6. 6Being wrong, on purpose11′
    7. 7What the model looks at9′
    8. 8The weighing machine10′
    9. 9Bigger data, more layers9′
    10. 10Where it helps, where it breaks12′
    1. 1The cell that fires9′
    2. 2Not every input matters10′
    3. 3A neuron is a line11′
    4. 4The rule that learns12′
    5. 5It always stops, if it can12′
    6. 6Two gates fall, one does not12′
    7. 7The first winter13′
    8. 8Why activations exist14′
    9. 9A neuron is a detector11′
    10. 10Two neurons beat XOR13′
    11. 11Build one12′
    12. 12Where they live today12′
    1. 1What people mean by AI7′
    2. 2Rules written by hand8′
    3. 3Thinking ahead9′
    4. 4The shortest way there8′
    5. 5Experts in a box8′
    6. 6The knowledge bottleneck8′
    7. 7Two winters8′
    8. 8Learning from examples instead7′
    9. 9Games as milestones9′
    10. 10Data and chips8′
    11. 11Attention and scale8′
    12. 12Chat goes public7′
    13. 13AI in the Arab world9′
    14. 14What it can and cannot do8′
    15. 15Reading AI news without the hype8′
    1. 1A photo is one point8′
    2. 2Arrows and lists8′
    3. 3Adding and stretching8′
    4. 4How long, how far8′
    5. 5One number for agreement9′
    6. 6Agreement without volume9′
    7. 7A matrix moves space9′
    8. 8Doing it by hand9′
    9. 9Doing it twice9′
    10. 10Two arrows are enough9′
    11. 11A strange place to live10′
    12. 12Where each idea lands9′
    1. 1The question every model asks7′
    2. 2How fast, not how much7′
    3. 3Zoom in until it is straight8′
    4. 4The derivative is a slope at a point9′
    5. 5The rules you will actually use8′
    6. 6The chain rule is the whole game10′
    7. 7One knob at a time8′
    8. 8The gradient points uphill9′
    9. 9Bottoms, tops and saddles9′
    10. 10Every model is a derivative machine9′
    11. 11Measure it both ways9′
    12. 12A taste of area8′
    1. 1Loss is a landscape10′
    2. 2The slope tells you which way is down11′
    3. 3One step downhill12′
    4. 4Too big, too small12′
    5. 5Bowls and real landscapes12′
    6. 6The wrong fear13′
    7. 7Noisy steps beat perfect ones13′
    8. 8Give the ball some weight12′
    9. 9A different step for every weight12′
    10. 10Adam: Both ideas at once13′
    11. 11Change the step as you go12′
    12. 12The spike that ruins a week12′
    13. 13Knowing when to stop13′
    14. 14Watch them race12′
    1. 1Fit it by eye first10′
    2. 2Why we square the error10′
    3. 3The loss surface is a bowl10′
    4. 4The exact answer11′
    5. 5The same answer, step by step11′
    6. 6More than one input11′
    7. 7Put the features on one scale10′
    8. 8Curves from a linear model11′
    9. 9Trying too hard12′
    10. 10Ridge and lasso as a budget12′
    11. 11R squared and its traps11′
    12. 12Where this lives inside every network11′
    1. 1A number you do not know yet7′
    2. 2Coins and categories9′
    3. 3The bell and the flat line10′
    4. 4Where it sits and how much it wobbles9′
    5. 5Joint, marginal, conditional10′
    6. 6Bayes is a belief update12′
    7. 7When knowing one tells you nothing7′
    8. 8Sampling and the law of large numbers10′
    9. 9Which setting explains the data best10′
    10. 10Surprise, measured10′
    11. 11Cross-entropy: The loss you will keep using11′
    12. 12KL divergence10′
    13. 13Softmax and temperature11′
    1. 1From a line to a label8′
    2. 2The soft switch8′
    3. 3Logistic regression10′
    4. 4The loss that punishes confidence9′
    5. 5Where the numbers came from10′
    6. 6Boundaries you can see12′
    7. 7The model that never trains9′
    8. 8The four boxes9′
    9. 9Caught and cried wolf10′
    10. 10Choosing a cut from money10′
    11. 11Every threshold at once10′
    12. 12When the answer is almost always no9′
    13. 13More than two answers9′
    14. 14The last layer of everything8′
    1. 1Every model is a graph11′
    2. 2The forward pass10′
    3. 3Every node knows one small derivative12′
    4. 4Chaining backwards12′
    5. 5One neuron, by hand11′
    6. 6Two layers, by hand12′
    7. 7Backprop in matrices11′
    8. 8Reverse-mode autodiff10′
    9. 9Why it costs so little9′
    10. 10Gradients that fade10′
    11. 11Gradients that explode, units that die10′
    12. 12Autograd, for real12′
    13. 13Checking gradients numerically11′
    14. 14What backprop is not11′
    1. 1Layers fold space14′
    2. 2Folds made of ReLU13′
    3. 3One layer can do anything13′
    4. 4Depth is cheaper than width14′
    5. 5Features nobody designed13′
    6. 6Where the weights start matters12′
    7. 7Batch norm and layer norm14′
    8. 8Dropout and weight decay13′
    9. 9The shortcut that opened up depth14′
    10. 10Five lines, in order14′
    11. 11Memorizing versus understanding14′
    12. 12Double descent13′
    13. 13Build an MLP on moons15′
    14. 14Reading the curves14′

Stage 2 · Intermediate

Inside the Models

Open up the networks that see, remember and write.

You pick up the working tools and look inside the transformer. By the end you know what happens inside a chat model.

    1. 1An image is a tensor11′
    2. 2One filter, dragged across13′
    3. 3Kernels you can draw13′
    4. 4Padding and stride13′
    5. 5Pooling12′
    6. 6How much can this neuron see?14′
    7. 7Channels and depth14′
    8. 8What actually won in 201214′
    9. 9Looking inside a vision model14′
    10. 10Going deep with ResNet13′
    11. 11Reusing a trained model14′
    12. 12When attention came for vision13′
    13. 13A panda, one nudge, a gibbon13′
    1. 1What a program is7′
    2. 2Values and names8′
    3. 3Numbers and text8′
    4. 4Lists8′
    5. 5Do this for every item9′
    6. 6Choosing8′
    7. 7Named recipes9′
    8. 8Reading an error without fear8′
    9. 9A grid of numbers10′
    10. 10Math on everything at once9′
    11. 11Draw it before you trust it8′
    12. 12One small experiment9′
    13. 13Reading other people's code9′
    1. 1Data where order matters8′
    2. 2A vector that remembers9′
    3. 3Unrolling the loop10′
    4. 4Backprop through time11′
    5. 5The same problem, now in time10′
    6. 6Gates that choose what to keep12′
    7. 7GRU: Fewer gates, similar result8′
    8. 8Training vs generating9′
    9. 9A character-level language model11′
    10. 10One vector for a whole sentence10′
    11. 11The case for attention9′
    1. 1Text has to become numbers8′
    2. 2Characters, words, subwords9′
    3. 3Byte pair encoding, merge by merge11′
    4. 4How big should the vocabulary be?9′
    5. 5Special tokens and chat templates8′
    6. 6Embeddings are learned coordinates11′
    7. 7Similarity and analogies10′
    8. 8Meaning from company10′
    9. 9Where a token sits10′
    10. 10The weird failures11′
    11. 11What Arabic costs a tokenizer12′
    12. 12Counting tokens and money8′
    1. 1A dictionary with fuzzy keys10′
    2. 2Queries, keys and values11′
    3. 3Scaled dot-product attention11′
    4. 4Softmax over positions10′
    5. 5Many heads, many questions11′
    6. 6No peeking ahead9′
    7. 7Reading attention maps10′
    8. 8The KV cache11′
    9. 9Why long context is expensive10′
    10. 10Cheaper attention variants12′
    11. 11Flash attention10′
    1. 1Anatomy of one block10′
    2. 2The feed-forward layer10′
    3. 3Norms and residuals10′
    4. 4The residual stream10′
    5. 5Positions in practice10′
    6. 6Three shapes of transformer11′
    7. 7GPT and BERT10′
    8. 8Where facts live10′
    9. 9Mixture of experts11′
    10. 10From vector to next token9′
    11. 11Why it beat the RNN10′
    12. 12Reading a real config11′
    13. 13Build a tiny transformer12′
    1. 1Nobody is average7′
    2. 2Shapes real data takes9′
    3. 3Two columns that move together9′
    4. 4Who is missing from the sample9′
    5. 5Luck has a size9′
    6. 6The interval is the result9′
    7. 7Pull the test set up by itself10′
    8. 8Could luck have done it11′
    9. 9Right nineteen times in twenty10′
    10. 10Keep trying and you will win10′
    11. 11Decide the size before you look10′
    12. 12Reading a table with suspicion11′
    1. 1Yesterday is a clue6′
    2. 2Three things in one line7′
    3. 3The clocks inside a series7′
    4. 4Tomorrow looks like today8′
    5. 5Keeping score7′
    6. 6Average the recent past8′
    7. 7Let old days fade8′
    8. 8The series predicts itself9′
    9. 9Two calendars, one shop9′
    10. 10No peeking8′
    11. 11Walk forward9′
    12. 12A range, not a number8′
    13. 13Big models against the baseline8′
    14. 14Surprises8′
    15. 15When not to forecast6′
    1. 1Rows, features, labels9′
    2. 2Pictures and words as rows9′
    3. 3Where labels come from10′
    4. 4Cleaning without lying10′
    5. 5The holes in the table10′
    6. 6Three piles10′
    7. 7The answer that slipped in10′
    8. 8When tomorrow is different11′
    9. 9The one case accuracy hides10′
    10. 10More data without new facts9′
    11. 11Who is in the table?10′
    12. 12The card that ships with the data10′
    1. 1A tree of questions8′
    2. 2Which question to ask first10′
    3. 3Why tree boundaries are staircases8′
    4. 4A tree that memorizes9′
    5. 5Pruning and depth limits9′
    6. 6Many trees, many samples9′
    7. 7Random forests10′
    8. 8Learning from your own mistakes10′
    9. 9Gradient boosting11′
    10. 10Feature importance and its traps9′
    11. 11Crediting each feature fairly10′
    12. 12When trees beat deep learning10′
    1. 1Near means similar8′
    2. 2Who is nearest8′
    3. 3k-means, step by step10′
    4. 4Where k-means breaks11′
    5. 5How many groups are there?8′
    6. 6Clusters inside clusters9′
    7. 7Density, not centers9′
    8. 8PCA is a rotation11′
    9. 9Maps of high-dimensional data10′
    10. 10Points that belong nowhere9′
    11. 11A grid with less in it10′
    12. 12The curse of dimensionality9′
    1. 1The score that lies8′
    2. 2Beat the dumbest thing first8′
    3. 3One metric per task11′
    4. 4Every example gets a turn10′
    5. 5Splits that respect reality9′
    6. 6Too stiff or too eager10′
    7. 7What the curve tells you to do10′
    8. 8Searching without fooling yourself11′
    9. 9The wobble between runs9′
    10. 10Look at the mistakes10′
    11. 11Hunting data leakage11′
    12. 12Benchmarks wear out9′
    13. 13Reporting results honestly9′
    1. 1Too much to show8′
    2. 2Near tastes9′
    3. 3People like you10′
    4. 4Two thin tables9′
    5. 5Fitting only what you know11′
    6. 6A map of taste9′
    7. 7The dish nobody has rated9′
    8. 8A click is not a like9′
    9. 9Find, then rank9′
    10. 10Scoring a list10′
    11. 11Offline wins, online losses9′
    12. 12The feed that feeds itself10′
    13. 13Varied, new and fair10′
    14. 14One space for search and suggestions9′
    1. 1Reading a log-log plot10′
    2. 2Power laws everywhere10′
    3. 3Compute, data, parameters9′
    4. 4The Kaplan result10′
    5. 5Chinchilla changes the recipe11′
    6. 6Spending a compute budget10′
    7. 7Emergence and its critics11′
    8. 8Why bigger keeps working10′
    9. 9What a frontier run costs10′
    10. 10Scaling at inference time9′
    11. 11What the curve does not predict10′
    1. 1Nobody gives it the answer8′
    2. 2Stay or try something new8′
    3. 3Mostly greedy, sometimes curious9′
    4. 4Where you stand changes the choice9′
    5. 5A dollar today9′
    6. 6How good is this square?9′
    7. 7Learning one step at a time11′
    8. 8Curious early, careful late9′
    9. 9Too many squares for a table11′
    10. 10Push up what worked10′
    11. 11A coach beside the player10′
    12. 12It learned the score, not the task8′
    13. 13Playing against yourself10′
    14. 14From games to robots9′
    15. 15The same loop, pointed at words8′
    1. 1Guess the next token9′
    2. 2Grading the guess10′
    3. 3Where the text comes from10′
    4. 4Cleaning the text10′
    5. 5Windows, batches, packing9′
    6. 6The loop at scale10′
    7. 7Warmup, then decay10′
    8. 8Small numbers, big spikes10′
    9. 9Splitting the work10′
    10. 10Tokens against parameters11′
    11. 11From predictor to assistant10′
    12. 12Learning from preferences11′
    13. 13DPO, and doing less10′
    14. 14Reading a benchmark table10′
    1. 1Where a prompt goes9′
    2. 2Three voices, one document10′
    3. 3Say the thing you want10′
    4. 4Show, do not tell11′
    5. 5Room to think11′
    6. 6One dial called temperature11′
    7. 7Cutting the tail10′
    8. 8Output you can parse10′
    9. 9Lost in the middle9′
    10. 10Why models make things up11′
    11. 11Prompt injection is a security bug11′
    12. 12Test sets, not vibes9′
    13. 13When prompting is the wrong tool9′

Stage 3 · Expert

Build and Discover

Build real systems, then explore the open edge of the field.

You build with models, train your own, and rebuild landmark papers. By the end you can pick an open problem and start on it.

    1. 1Why the model does not know your data8′
    2. 2Search by meaning10′
    3. 3Chunking decides everything11′
    4. 4Vector indexes10′
    5. 5Keywords still matter10′
    6. 6Read the shortlist properly9′
    7. 7Answers with receipts9′
    8. 8Retrieval in Arabic10′
    9. 9Measuring the two halves11′
    10. 10Four ways it fails10′
    11. 11Paste it all, or go and find it9′
    12. 12Teach it, or show it9′
    13. 13Every tool is a retriever8′
    1. 1The loop9′
    2. 2Tools are functions with descriptions10′
    3. 3Tool output is data, not orders10′
    4. 4Planning10′
    5. 5Memory between steps10′
    6. 6More than one agent10′
    7. 7Tool protocols9′
    8. 8Guardrails and permission models11′
    9. 9Loops, hallucinated tools and stalls10′
    10. 10Evaluating agents11′
    11. 11Keeping the bill sane9′
    12. 12What they can do today10′
    1. 1One guess is not enough8′
    2. 2Steps on the page9′
    3. 3Ask again, take the vote9′
    4. 4Thinking time or a bigger model10′
    5. 5A checker picks the best10′
    6. 6Grade the answer or the working10′
    7. 7Rewards you can check11′
    8. 8What the training grew10′
    9. 9The reasoning models9′
    10. 10Searching a tree of thoughts10′
    11. 11Tools in the middle of a thought9′
    12. 12Teaching small models to think9′
    13. 13When thinking goes wrong10′
    14. 14Measuring reasoning honestly11′
    1. 1Anatomy of a request9′
    2. 2Where the key lives8′
    3. 3Streaming10′
    4. 4Structured outputs9′
    5. 5Tool calling in production11′
    6. 6Prompt caching10′
    7. 7Cost and latency budgets10′
    8. 8Retries and fallbacks10′
    9. 9Seeing what happened10′
    10. 10Evals in CI11′
    11. 11Choosing a model9′
    1. 1The model is ten percent8′
    2. 2Two phases and a cache10′
    3. 3Batching and the tail11′
    4. 4The cost of one answer10′
    5. 5Watching quality, not uptime10′
    6. 6When the world moves10′
    7. 7Loops that help, loops that rot10′
    8. 8Guardrails and fallbacks10′
    9. 9Shipping a change10′
    10. 10Versions you can roll back9′
    11. 11What you keep9′
    12. 12When not to use AI9′
    1. 1A task worth a small model9′
    2. 2A few hundred labels10′
    3. 3The number you have to beat9′
    4. 4Training a tiny classifier12′
    5. 5Overfit on purpose10′
    6. 6Three ways to fix it11′
    7. 7A language model you can read12′
    8. 8Fine-tune or start from scratch10′
    9. 9LoRA, by picture10′
    10. 10Letting the giant teach10′
    11. 11Shrinking it to ship10′
    12. 12Judging it against the giant11′
    1. 1Who gets the answer9′
    2. 2The wall is memory10′
    3. 3One scale or many11′
    4. 4One number ruins the row11′
    5. 5Round, then repair12′
    6. 6A teacher with a vocabulary11′
    7. 7Cutting weights11′
    8. 8One base, many adapters10′
    9. 9Cheap to run, costly to hold10′
    10. 10A small model drafts10′
    11. 11How far ahead to guess10′
    12. 12The long conversation bill10′
    13. 13A model in your pocket10′
    14. 14Choosing with a measured eval11′
    1. 1Squeeze, then rebuild10′
    2. 2Walking through latent space11′
    3. 3Variational autoencoders12′
    4. 4Two networks in a fight12′
    5. 5Why GANs collapse12′
    6. 6Destroying an image on purpose12′
    7. 7Learning to undo one step13′
    8. 8Following the data uphill12′
    9. 9Sampling schedules11′
    10. 10Conditioning and guidance13′
    11. 11Diffusion in latent space12′
    12. 12From words to pictures12′
    13. 13Making time consistent12′
    14. 14Scoring a picture maker12′
    15. 15What these models understand12′
    1. 1Every sense becomes a line9′
    2. 2Pictures meet their captions8′
    3. 3Right pairs up, wrong pairs down10′
    4. 4Classify by writing captions9′
    5. 5Pictures inside a language model10′
    6. 6What a picture costs9′
    7. 7Reading a page11′
    8. 8Sound as a picture10′
    9. 9From voice to words10′
    10. 10Words back to voice10′
    11. 11Pictures in time10′
    12. 12Making, not only reading8′
    13. 13Where they fail10′
    14. 14Models that act11′
    1. 1What counts as a feature10′
    2. 2Neurons, features, circuits11′
    3. 3The neuron that means five things10′
    4. 4Superposition12′
    5. 5Sparse autoencoders13′
    6. 6Probing11′
    7. 7Activation patching13′
    8. 8Induction heads12′
    9. 9Attribution maps and their failures11′
    10. 10What we cannot read yet11′
    11. 11Why safety needs this12′
    1. 1What alignment means10′
    2. 2Doing exactly what you asked10′
    3. 3Searching harder makes it worse11′
    4. 4The model learns your taste10′
    5. 5Confident and wrong10′
    6. 6Answer or stay quiet11′
    7. 7Bias you can measure11′
    8. 8What the model remembers10′
    9. 9Why a refusal breaks11′
    10. 10Going looking on purpose11′
    11. 11An evaluation you can trust11′
    12. 12Looking inside11′
    13. 13Misuse, accidents and uplift10′
    14. 14The rules, in plain words10′
    15. 15Electricity, work and what nobody knows10′
    1. 1Scoping a project you will finish10′
    2. 2Project 1: A spam filter12′
    3. 3Project 1: The cut is a decision11′
    4. 4Project 2: A digit recognizer12′
    5. 5Project 2: Where it confuses itself11′
    6. 6Project 3: A tiny next-word model13′
    7. 7Project 4: An answerer that cites12′
    8. 8Project 4: Saying I do not know11′
    9. 9Project 5: Catching a regression12′
    10. 10Project 6: Redraw one figure12′
    11. 11Writing it up10′
    1. 1Why read papers at all7′
    2. 2Preprints, venues and peer review8′
    3. 3The shape of a paper9′
    4. 4Abstract, then figures, then decide11′
    5. 5Reading the math11′
    6. 6Reading the results table11′
    7. 7Spotting a weak baseline10′
    8. 8Following the thread9′
    9. 9Reproducing one claim13′
    10. 10Spotting overclaiming9′
    11. 11Writing a summary10′
    12. 12Keeping a reading list8′
    1. 1Perceptron (1958)11′
    2. 2Learning Representations by Back-Propagating Errors (1986)12′
    3. 3LeNet (1998)11′
    4. 4AlexNet (2012)11′
    5. 5Word2Vec (2013)10′
    6. 6Sequence to Sequence (2014)10′
    7. 7Attention Is All You Need (2017)13′
    8. 8GPT-2 and GPT-311′
    9. 9Scaling Laws (2020)10′
    10. 10Chinchilla (2022)10′
    11. 11CLIP (2021)11′
    12. 12Denoising Diffusion (2020)12′
    13. 13InstructGPT (2022)11′
    14. 14DeepSeek V2 and V311′
    1. 1What counts as an open problem8′
    2. 2Where does scaling stop?13′
    3. 3Running out of text12′
    4. 4Do models reason?13′
    5. 5Learning after training12′
    6. 6Evaluation is broken11′
    7. 7Can we ever read a model?11′
    8. 8Energy, chips and limits10′
    9. 9Alignment as an open problem12′
    10. 10Senses and bodies11′
    11. 11Picking a first research project11′
    12. 12Running it without fooling yourself12′