R3 · Open Problems
Energy, Chips and Limits
Put current training energy in context and name the physical constraints.
Training GPT-3 used about 1,287 megawatt hours of electricity. The world's data centers used about 460 terawatt hours in 2022 alone. Those two numbers are not close.