Hierarchically structured neural networks: A way to shape a ‘magma’ of neurons

Abstract In this paper we present a new way of structuring standard classes of NN, so obtaining a new class of parametric functions, which will be named ‘Hierarchically-Structured-Neural-Networks’ (HSNNs). HSNNs are a special class of networks, constituted by two sub-networks: the ‘slave’ unit and the ‘master’ unit; the master network is fed by a subset of inputs and its outputs are used to ‘drive’ the parameters of the slave network, whose inputs are disjoint from those of the other sub-network. After the general definition of HSNN has been given, two simple classes of HSNNs are presented, and dedicated Back-Propagation algorithms are derived. The HSNNs are a useful tool when some prior knowledge of the nonlinear function to be approximated or designed is available; this is illustrated by means of five examples, where a variety of simple problems are discussed.

[1]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[2]  Yung C. Shin,et al.  Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems , 1994, IEEE Trans. Neural Networks.

[3]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[4]  Michael L. Mavrovouniotis,et al.  Hierarchical neural networks , 1992 .

[5]  Xin Feng,et al.  Artificial neural network identification of partially known dynamic nonlinear systems , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[6]  Jack L. Meador,et al.  Encoding a priori information in feedforward networks , 1991, Neural Networks.

[7]  Lee W. Johnson,et al.  Numerical Analysis , 1977 .

[8]  Thomas Parisini,et al.  Neural networks for feedback feedforward nonlinear control systems , 1994, IEEE Trans. Neural Networks.

[9]  Christopher M. Bishop,et al.  Curvature-driven smoothing: a learning algorithm for feedforward networks , 1993, IEEE Trans. Neural Networks.

[10]  J. Sjöberg.  Non-Linear System Identification with Neural Networks , 1995 .

[11]  Luigi Piroddi,et al.  Minimum variance control of a class of nonlinear plants with neural networks , 1993 .

[12]  Luigi Piroddi,et al.  GMV technique for nonlinear control with neural networks , 1994 .

[13]  Allon Guez,et al.  The role of a priori knowledge of plant dynamics in neurocontroller design , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[14]  S. Bittanti,et al.  Neural implementation of GMV control schemes based on affine input/output models , 1997 .

[15]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[16]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[17]  Sergio M. Savaresi,et al.  A Neural-Network Simulator for the 3-Lobes Stern Seal of a S.E.S , 1997, Intell. Autom. Soft Comput..

[18]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .