The Chebyshev-polynomials-based unified model neural networks for function approximation

In this paper, we propose the approximate transformable technique, which includes the direct transformation and indirect transformation, to obtain a Chebyshev-Polynomials-Based (CPB) unified model neural networks for feedforward/recurrent neural networks via Chebyshev polynomials approximation. Based on this approximate transformable technique, we have derived the relationship between the single-layer neural networks and multilayer perceptron neural networks. It is shown that the CPB unified model neural networks can be represented as a functional link networks that are based on Chebyshev polynomials, and those networks use the recursive least square method with forgetting factor as learning algorithm. It turns out that the CPB unified model neural networks not only has the same capability of universal approximator, but also has faster learning speed than conventional feedforward/recurrent neural networks. Furthermore, we have also derived the condition such that the unified model generating by Chebyshev polynomials is optimal in the sense of error least square approximation in the single variable ease. Computer simulations show that the proposed method does have the capability of universal approximator in some functional approximation with considerable reduction in learning time.

[1]  Y. H. Pao,et al.  Characteristics of the functional link net: a higher order delta rule net , 1988, IEEE 1988 International Conference on Neural Networks.

[2]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[3]  M. Farsi,et al.  Non‐linear system identification and control based on neural and self‐tuning control , 1993 .

[4]  Toshio Fukuda,et al.  An adaptive control for CARMA systems using linear neural networks , 1992 .

[5]  David Elliott,et al.  Error of truncated Chebyshev series and other near minimax polynomial approximations , 1987 .

[6]  Michael T. Manry,et al.  Conventional modeling of the multilayer perceptron using polynomial basis functions , 1993, IEEE Trans. Neural Networks.

[7]  J. D. Cole.  A new derivation of a closed form expression for Chebyshev polynomials of any order , 1989 .

[8]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[9]  Yung C. Shin,et al.  Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems , 1994, IEEE Trans. Neural Networks.

[10]  Anastasios N. Venetsanopoulos,et al.  Artificial neural networks - learning algorithms, performance evaluation, and applications , 1992, The Kluwer international series in engineering and computer science.

[11]  Henk B. Verbruggen,et al.  Single-layer networks for nonlinear system identification , 1994 .

[12]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[13]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[14]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[15]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[16]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[17]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[18]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[19]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[20]  Hong Chen,et al.  Approximation capability in C(R¯n) by multilayer feedforward networks and related problems , 1995, IEEE Trans. Neural Networks.

[21]  H. White,et al.  Universal approximation using feedforward networks with non-sigmoid hidden layer activation functions , 1989, International 1989 Joint Conference on Neural Networks.

[22]  D. E. Brown,et al.  A polynomial network for predicting temperature distributions , 1994, IEEE Trans. Neural Networks.

[23]  Rolf Unbehauen,et al.  Canonical piecewise-linear networks , 1995, IEEE Trans. Neural Networks.

[24]  Roy Batruni,et al.  A multilayer neural network with piecewise-linear structure and back-propagation learning , 1991, IEEE Trans. Neural Networks.

[25]  Yagyensh C. Pati,et al.  Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations , 1993, IEEE Trans. Neural Networks.

[26]  J.-N. Lin,et al.  Canonical representation: from piecewise-linear function to piecewise-smooth functions , 1993 .

[27]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[28]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[29]  Nader Sadegh,et al.  A perceptron network for functional identification and control of nonlinear systems , 1993, IEEE Trans. Neural Networks.

[30]  Dejan J. Sobajic,et al.  Neural-net computing and the intelligent control of systems , 1992 .

[31]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[32]  Colin Giles,et al.  Learning, invariance, and generalization in high-order neural networks. , 1987, Applied optics.

[33]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[34]  R. Chapman,et al.  Chebyshev-polynomial-based Schur algorithm , 1990 .

[35]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[36]  V. Etxebarria,et al.  Adaptive control of discrete systems using neural networks , 1994 .

[37]  Stephen A. Billings,et al.  Please Scroll down for Article , 1992 .

[38]  Donald F. Specht,et al.  Probabilistic neural networks and the polynomial Adaline as complementary techniques for classification , 1990, IEEE Trans. Neural Networks.