Supervised Learning: Can it Escape its Local Minimum?

Tremendous progress and new research horizons have opened up in developing artificial neural network (ANN) systems which use supervised learning as an elementary building block in performing more brain-like, generic tasks such as control or planning or forecasting across time [1, 2]. However, many people believe that research into supervised learning itself is stuck in a kind of local minimum or plateau. Limited capabilities in supervised learning have created a serious dilemma, forcing us to choose between fast real-time learning versus good generalization with many inputs, without allowing us to have both together.