The dependence identification neural network construction algorithm

An algorithm for constructing and training multilayer neural networks, dependence identification, is presented in this paper. Its distinctive features are that (i) it transforms the training problem into a set of quadratic optimization problems that are solved by a number of linear equations, (ii) it constructs an appropriate network to meet the training specifications, and (iii) the resulting network architecture and weights can be further refined with standard training algorithms, like backpropagation, giving a significant speedup in the development time of the neural network and decreasing the amount of trial and error usually associated with network development.

[1]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[2]  H. Elman.  Iterative methods for large, sparse, nonsymmetric systems of linear equations , 1982 .

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  R. Hecht-Nielsen.  Counterpropagation networks. , 1987, Applied optics.

[5]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..

[6]  Gerald Tesauro,et al.  Scaling Relationships in Back-propagation Learning , 1988, Complex Syst..

[7]  Alberto L. Sangiovanni-Vincentelli,et al.  Efficient Parallel Learning Algorithms for Neural Networks , 1988, NIPS.

[8]  Richard T. Scalettar,et al.  Emergence of grandmother memory in feed forward networks: learning with noise and forgetfulness , 1988 .

[9]  Geoffrey E. Hinton,et al.  Learning distributed representations of concepts. , 1989 .

[10]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[11]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[12]  M. Golea,et al.  A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons , 1990 .

[13]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[14]  L. Jones.  Constructive approximations for neural networks by sigmoidal functions , 1990, Proc. IEEE.

[15]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[16]  Panos J. Antsaklis,et al.  A simple method to derive bounds on the size and to train multilayer neural networks , 1991, IEEE Trans. Neural Networks.

[17]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[18]  Panos J. Antsaklis,et al.  Neural network training via quadratic optimization , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[19]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1994 .

[20]  Panos J. Antsaklis,et al.  The dependence identification neural network construction algorithm , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).