Stage 3 · Expert · R3
Open Problems
Nobody knows the answers in this chapter, and that is the point.
12 lessons · 136 minResearch
About this chapter
Every other chapter taught you something the field has settled. This one maps what is still open, and it is strict about which arguments carry evidence and which carry only confidence. You will see the strongest case both ways on scaling, data, reasoning, memory, evaluation, interpretability, energy, alignment and bodies, and the experiment that would close each one. It ends with how to pick a first research question small enough to finish and real enough to matter.
What you will be able to do
- 18 min
What Counts as an Open Problem
Tell an open problem apart from an engineering task and a matter of taste.
- 213 min
Where Does Scaling Stop?
Lay out the strongest case for and against scaling continuing to work.
- 312 min
Running Out of Text
Estimate available high-quality data and evaluate the proposed escapes.
- 413 min
Do Models Reason?
Describe an experiment that would tell reasoning apart from retrieval.
- 512 min
Learning After Training
Explain catastrophic forgetting and why a permanent memory is still unsolved.
- 611 min
Evaluation Is Broken
Explain contamination and saturation, and what a better benchmark needs.
- 711 min
Can We Ever Read a Model?
State what a full mechanistic explanation would require and how far we are.
- 810 min
Energy, Chips and Limits
Put current training energy in context and name the physical constraints.
- 912 min
Alignment as an Open Problem
Explain scalable oversight and why current methods may not extend.
- 1011 min
Senses and Bodies
Say why a model that reads everything still cannot reliably pick up a cup.
- 1111 min
Picking a First Research Project
Choose a small, testable question and write the one-page plan for it.
- 1212 min
Running It Without Fooling Yourself
Run the comparison with seeds and a baseline, and read the result honestly.