A Novel Learning Scheme for Chebyshev Functional Link Neural Networks

A hybrid learning scheme (ePSO-BP) to train Chebyshev Functional Link Neural Network (CFLNN) for classification is presented. The proposed method is referred as hybrid CFLNN (HCFLNN). The HCFLNN is a type of feed-forward neural networks which have the ability to transform the nonlinear input space into higher dimensional-space where linear separability is possible. Moreover, the proposed HCFLNN combines the best attribute of particle swarm optimization (PSO), back propagation learning (BP learning), and functional link neural networks (FLNNs). The proposed method eliminates the need of hidden layer by expanding the input patterns using Chebyshev orthogonal polynomials. We have shown its effectiveness of classifying the unknown pattern using the publicly available datasets obtained from UCI repository. The computational results are then compared with functional link neural network (FLNN) with a generic basis functions, PSO-based FLNN, and EFLN. From the comparative study, we observed that the performance of the HCFLNN outperforms FLNN, PSO-based FLNN, and EFLN in terms of classification accuracy.

[1]  Ganapati Panda,et al.  Identification of nonlinear dynamic systems using functional link artificial neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Yuval Davidor,et al.  Epistasis Variance: Suitability of a Representation to Genetic Algorithms , 1990, Complex Syst..

[3]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[4]  Lifeng Xi,et al.  Evolving artificial neural networks using an improved PSO and DPSO , 2008, Neurocomputing.

[5]  Wan-De Weng,et al.  A channel equalizer using reduced decision feedback Chebyshev functional link artificial neural networks , 2007, Inf. Sci..

[6]  J C Patra,et al.  Modeling of an intelligent pressure sensor using functional link artificial neural networks. , 2000, ISA transactions.

[7]  Ye Tian,et al.  An Improved Particle Swarm Algorithms for Global Optimization , 2010, 2010 International Conference on Machine Vision and Human-machine Interface.

[8]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[9]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[10]  José A. Macías,et al.  Evolution of functional link networks , 2001, IEEE Trans. Evol. Comput..

[11]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[12]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[13]  Joost N. Kok,et al.  Finding Functional Links for Neural Networks by Evolutionary Computation , 2007 .

[14]  Y. Takefuji,et al.  Functional-link net computing: theory, system architecture, and functionalities , 1992, Computer.

[15]  Banshidhar Majhi,et al.  An Improved Scheme for Digital Watermarking Using Functional Link Artificial Neural Network , 2005 .

[16]  A. C. Liew,et al.  A functional-link-neural network for short-term electric load forecasting , 1999, J. Intell. Fuzzy Syst..

[17]  Jagdish C. Patra,et al.  A functional link artificial neural network for adaptive channel equalization , 1995, Signal Process..

[18]  Alex ChiChung Kot,et al.  Nonlinear dynamic system identification using Chebyshev functional link artificial neural networks , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[19]  Indra Narayan Kar,et al.  On-line system identification of complex systems using Chebyshev neural networks , 2007, Appl. Soft Comput..

[20]  Sung-Bae Cho,et al.  Genetic Feature Selection for Optimal Functional Link Artificial Neural Network in Classification , 2008, IDEAL.

[21]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[22]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[23]  Teodor Marcu,et al.  Dynamic functional-link neural networks genetically evolved applied to system identification , 2004, ESANN.

[24]  B. B. Misra,et al.  Functional Link Artificial Neural Network for Classification Task in Data Mining , 2007 .

[25]  Ge Xiurun,et al.  An improved PSO-based ANN with simulated annealing technique , 2005, Neurocomputing.

[26]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[27]  Olvi L. Mangasarian,et al.  Nonlinear Knowledge-Based Classification , 2008, IEEE Transactions on Neural Networks.

[28]  Robert W. Newcomb,et al.  Planning with a functional neural network architecture , 1999, IEEE Trans. Neural Networks.

[29]  Subhas Chandra Mukhopadhyay,et al.  Functional link neural network-based intelligent sensors for harsh environments , 2008 .

[30]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[31]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[32]  Colin Giles,et al.  Learning, invariance, and generalization in high-order neural networks. , 1987, Applied optics.

[33]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[34]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[35]  Y. H. Pao,et al.  Characteristics of the functional link net: a higher order delta rule net , 1988, IEEE 1988 International Conference on Neural Networks.

[36]  Joost N. Kok,et al.  Feature Selection for Neural Networks through Functional Links Found by Evolutionary Computation , 1997, IDA.

[37]  P. Fourie,et al.  The particle swarm optimization algorithm in size and shape optimization , 2002 .

[38]  Issam Abu-Mahfouz,et al.  A comparative study of three artificial neural networks for the detection and classification of gear faults , 2005, Int. J. Gen. Syst..

[39]  Joydeep Ghosh,et al.  Efficient Higher-Order Neural Networks for Classification and Function Approximation , 1992, Int. J. Neural Syst..

[40]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[41]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[42]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[43]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[44]  K. N. Srivastava,et al.  Degree of insecurity estimation in a power system using functional link neural network , 2002 .

[45]  Orly Yadid-Pecht,et al.  Modified high-order neural network for invariant pattern recognition , 2005, Pattern Recognit. Lett..

[46]  Albert A. Groenwold,et al.  A Study of Global Optimization Using Particle Swarms , 2005, J. Glob. Optim..

[47]  Awang Bono,et al.  Application of ultrasonic waves coupled with functional link neural network for estimation of carrageenan concentration , 2008 .