D7 · Neural Scaling Laws
What a Frontier Run Costs
Estimate the compute and money behind a large training run.
Meta published the bill for its largest 2024 model: 30.8 million graphics card hours. On one card, that is about 3,500 years. Meta ran up to 16,384 cards at once, which brought it down to months. The power left about 8,930 tons of carbon dioxide, counted by where it was used.