Two-step approach in the training of regulated activation weight neural networks (RAWN)

Abstract Feedforward neural networks with a single hidden layer of neurons and a linear output layer are a convenient way to model a nonlinear input-output mapping. If the activation weights, i.e. the weights between input and hidden-layer neurons, are known, an estimation problem remains that is linear in the parameters. This can easily be solved by standard least-squares methods. The problem thus reduces to finding appropriate activation weights. This paper describes a method to obtain the activation weights, based on local linear approximations, which also can be solved with standard least-squares techniques. The local linear models can be obtained by fuzzy clustering methods. The method is demonstrated on a simple example. With the proposed method the weights are obtained very fast, and the results are good. The method is also flexible with respect to the incorporation of a priori process knowledge.

[1]  Dejan J. Sobajic,et al.  Neural-net computing and the intelligent control of systems , 1992 .

[2]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[3]  Petri A. Jokinen.  DYNAMICALLY CAPACITY ALLOCATING NETWORK MODELS FOR CONTINUOUS LEARNING , 1991 .

[4]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[5]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[6]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[7]  R.N. Dave.  Boundary detection through fuzzy clustering , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[8]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[9]  Henk B. Verbruggen,et al.  Artificial Intelligence in Real-Time Control , 1992 .

[10]  Hubert A.B. Te Braake,et al.  Random activation weight neural net (RAWN) for fast non-iterative training. , 1995 .

[11]  C. D. Boor,et al.  On Calculating B-splines , 1972 .

[12]  Stephen A. Billings,et al.  A comparison of the backpropagation and recursive prediction error algorithms for training neural networks , 1991 .

[13]  George Stephanopoulos,et al.  Chemical Process Control: An Introduction to Theory and Practice , 1983 .

[14]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[15]  Bedri C. Cetin,et al.  Terminal repeller unconstrained subenergy tunneling (trust) for fast global optimization , 1993 .

[16]  Bart Kosko,et al.  Neural networks and fuzzy systems: a dynamical systems approach to machine intelligence , 1991 .

[17]  Bart Kosko,et al.  Neural networks and fuzzy systems , 1998 .

[18]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[19]  Sandro Ridella,et al.  Statistically controlled activation weight initialization (SCAWI) , 1992, IEEE Trans. Neural Networks.

[20]  Nazif Tepedelenlioglu,et al.  A fast new algorithm for training feedforward neural networks , 1992, IEEE Trans. Signal Process..

[21]  Sheng Chen,et al.  Parallel recursive prediction error algorithm for training layered neural networks , 1990 .

[22]  Giovanni Muscato,et al.  Improving back-propagation learning using auxiliary neural networks , 1992 .

[23]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[24]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[25]  Henk B. Verbruggen,et al.  Design and real time testing of a neural model predictive controller for a nonlinear system , 1995 .

[26]  Sharad Singhal,et al.  Training feed-forward networks with the extended Kalman algorithm , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[27]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[28]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[29]  M. Schetzen.  The Volterra and Wiener Theories of Nonlinear Systems , 1980 .

[30]  R. H. Myers.  Classical and modern regression with applications , 1986 .

[31]  Amir Fijany,et al.  Learning without local minima , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[32]  Pierre Baldi,et al.  Gradient descent learning algorithm overview: a general dynamical systems perspective , 1995, IEEE Trans. Neural Networks.

[33]  Henk B. Verbruggen,et al.  Regulated Activation Weights Neural Network (RAWN) , 1996, ESANN.

[34]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[35]  Mark A. Kramer,et al.  Modeling chemical processes using prior knowledge and neural networks , 1994 .