Properties and performance of orthogonal neural network in function approximation

Backpropagation neural network has been applied successfully to solving uncertain problems in many fields. However, unsolved drawbacks still exist such as the problems of local minimum, slow convergence speed, and the determination of initial weights and the number of processing elements. In this paper, we introduce a single‐layer orthogonal neural network (ONN) that is developed based on orthogonal functions. Since the processing elements are orthogonal to one another and there is no local minimum of the error function, the orthogonal neural network is able to avoid the above problems. Among the five existing orthogonal functions, Legendre polynomials and Chebyshev polynomials of the first kind have the properties of recursion and completeness. They are the most suitable to generate the neural network. Some typical examples are given to show their performance in function approximation. The results show that ONN has excellent convergence performance. Moreover, ONN is capable of approximating the mathematic model of backpropagation neural network. Therefore, it should be able to be applied to various applications that backpropagation neural network is suitable to solve. © 2001 John Wiley & Sons, Inc.

[1]  Michael T. Manry,et al.  Neural subnet design by direct polynomial mapping , 1992, IEEE Trans. Neural Networks.

[2]  Ehud D. Karnin,et al.  A simple procedure for pruning back-propagation trained neural networks , 1990, IEEE Trans. Neural Networks.

[3]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[4]  Michael T. Manry,et al.  Conventional modeling of the multilayer perceptron using polynomial basis functions , 1993, IEEE Trans. Neural Networks.

[5]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[6]  Johan A. K. Suykens,et al.  Static and dynamic stabilizing neural controllers, applicable to transition between equilibrium points , 1994, Neural Networks.

[7]  Dejan J. Sobajic,et al.  Neural-net computing and the intelligent control of systems , 1992 .

[8]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[9]  Esther Levin,et al.  Neural network architecture for adaptive system modeling and control , 1991, International 1989 Joint Conference on Neural Networks.

[10]  Hideaki Sakai,et al.  A nonlinear regulator design in the presence of system uncertainties using multilayered neural network , 1991, IEEE Trans. Neural Networks.

[11]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[12]  F. W. Kellaway,et al.  Advanced Engineering Mathematics , 1969, The Mathematical Gazette.

[13]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.