D4 · Deep Learning
One Layer Can Do Anything
State universal approximation precisely and say what it does not promise.
One hidden layer, wide enough, can copy any smooth curve you can draw, as closely as you like. That was proved in 1989. Almost nobody has built a network that way since.