Learning Method Utilizing Singular Region of Multilayer Perceptron

In a search space of multilayer perceptron having J hidden units, MLP( J), there exists a singular flat region created by the projection of the optimal solution of MLP( J−1). Since such a singular region causes serious slowdown for learning methods, a method for avoiding the region has been aspired. However, such avoiding does not guarantee the quality of the final solution. This paper proposes a new learning method which does not avoid but makes good use of singular regions to find a solution good enough for MLP( J). The potential of the method is shown by our experiments using artificial data sets, XOR problem, and a real data set.