D4 · Deep Learning
Memorizing Versus Understanding
Detect memorization and choose between more data, more noise or less model.
A team shuffled every label in an image dataset at random. Cats became trucks. A standard network still reached zero training error. It had memorized 50,000 answers with no pattern behind them.