Stationary points of a single-layer perceptron for nonseparable data models

Abstract A single-layer perceptron divides the input signal space into two regions separated by a hyperplane. In many applications, however, the training signal of the adaptive algorithm represents more complicated decision regions that may not be linearly separable. For these cases, it is usually not obvious how the adaptive algorithm of a single-layer perceptron will perform, in terms of its convergence properties and the optimum location of the hyperplane boundary. In this paper, we determine the stationary points of Rosenblatt's learning algorithm for a single-layer perceptron and two nonseparable models of the training data. Our analysis is based on a system identification formulation of the training signal, and the perceptron input signals are modeled as independent Gaussian sequences. Expressions for the corresponding performance function are also derived, and computer simulations are presented that verify the analytical results.

[1]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[2]  J. J. Shynk,et al.  Steady-state analysis of a single-layer perceptron based on a system identification model with bias terms , 1991 .

[3]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .

[4]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[5]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[6]  John J. Shynk,et al.  Statistical analysis of the single-layer backpropagation algorithm. II. MSE and classification performance , 1993, IEEE Trans. Signal Process..

[7]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[8]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[9]  John J. Shynk,et al.  Statistical analysis of the single-layer backpropagation algorithm. I. mean weight behavior , 1993, IEEE Trans. Signal Process..

[10]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[11]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[12]  John J. Shynk,et al.  Convergence properties and stationary points of a perceptron learning algorithm , 1990 .

[13]  Robert F. Pawula,et al.  A modified version of Price's theorem , 1967, IEEE Trans. Inf. Theory.

[14]  John J. Shynk,et al.  Performance surfaces of a single-layer perceptron , 1990, IEEE Trans. Neural Networks.

[15]  Robert Price,et al.  A useful theorem for nonlinear devices having Gaussian inputs , 1958, IRE Trans. Inf. Theory.