Multilayer perceptron can be trained with empirical data to estimate general real-valued functions or to be used as a pattern classifier to estimate indicator functions. The typical backpropagation learning algorithm and its variations do not distinguish the training of an MLP as a pattern classifier from that of a general function estimator. In this paper, we present a learning algorithm based on an optimization layer by layer (OLL) procedure. Its main difference from previously reported OLL-type learning algorithms is that the weights between the last hidden layer and the output layer are determined through optimization of a piecewise linear objective function subject to constraints designed specifically for training an MLP to be a pattern classifier. The performance of the proposed learning algorithm is compared with that of the backpropagation algorithm, the modified Newton's method and the improved descending epsilon algorithm over multiple training sessions using both simulated and real data classification problems.
[1]
Dimitri P. Bertsekas,et al.
Nonlinear Programming
,
1997
.
[2]
Christopher M. Bishop,et al.
Neural networks for pattern recognition
,
1995
.
[3]
S. Ergezinger,et al.
An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer
,
1995,
IEEE Trans. Neural Networks.
[4]
Vladimir N. Vapnik,et al.
The Nature of Statistical Learning Theory
,
2000,
Statistics for Engineering and Information Science.
[5]
David G. Luenberger,et al.
Linear and nonlinear programming
,
1984
.