Classification by Evolutionary Generalized Radial Basis Functions

This paper proposes a novelty neural network model by using generalized kernel functions for the hidden layer of a feed forward network (Generalized Radial Basis Functions, GRBF), where the architecture, weights and node typology are learned through an evolutionary programming algorithm. This new kind of model is compared with the corresponding models with standard hidden nodes: Product Unit Neural Networks (PUNN), Multilayer Perceptrons (MLP) and the RBF neural networks. The methodology proposed is tested using six benchmark classification datasets from well-known machine learning problems. Generalized basis functions are found to present a better performance than the other standard basis functions for the task of classification.

[1]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[2]  Chee Peng Lim,et al.  A hybrid SOM-kMER model for data visualization and classification , 2005, Int. J. Hybrid Intell. Syst..

[3]  Michel Verleysen,et al.  On the Effects of Dimensionality on Data Analysis with Neural Networks , 2009, IWANN.

[4]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[5]  Pedro Antonio Gutiérrez,et al.  Combined projection and kernel basis functions for classification in evolutionary neural networks , 2009, Neurocomputing.

[6]  L.N. de Castro,et al.  An evolutionary clustering technique with local search to design RBF neural network classifiers , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[7]  Mark J. L. Orr,et al.  Regularization in the Selection of Radial Basis Function Centers , 1995, Neural Computation.

[8]  Carl G. Looney,et al.  Radial basis functional link nets and fuzzy reasoning , 2002, Neurocomputing.

[9]  John E. Moody,et al.  Fast adaptive k-means clustering: some empirical results , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[10]  David E. Rumelhart,et al.  Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks , 1989, Neural Computation.

[11]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[12]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[13]  Boleslaw K. Szymanski,et al.  Introduction to Scientific Data Mining: Direct Kernel Methods and Applications , 2004, Computationally Intelligent Hybrid Systems.

[14]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[15]  Rupert G. Miller.  Simultaneous Statistical Inference , 1966 .

[16]  Chris Bishop,et al.  Improving the Generalization Properties of Radial Basis Function Neural Networks , 1991, Neural Computation.

[17]  A. Tamhane,et al.  Multiple Comparison Procedures , 2009 .

[18]  Antonio J. Rivera,et al.  CO2RBFN for short-term forecasting of the extra virgin olive oil price in the Spanish market , 2010, Int. J. Hybrid Intell. Syst..

[19]  Kwang Bo Cho,et al.  Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction , 1996, Fuzzy Sets Syst..

[20]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[21]  Witold Pedrycz,et al.  Improving RBF networks performance in regression tasks by means of a supervised fuzzy clustering , 2006, Neurocomputing.

[22]  Lei Xu,et al.  RBF nets, mixture experts, and Bayesian Ying-Yang learning , 1998, Neurocomputing.

[23]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[24]  De-Shuang Huang,et al.  A mended hybrid learning algorithm for radial basis function neural networks to improve generalization capability , 2007 .

[25]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[26]  James D. Keeler,et al.  Layered Neural Networks with Gaussian Hidden Units as Universal Approximations , 1990, Neural Computation.

[27]  Damien François.  High-dimensional Data Analysis , 2008 .

[28]  Pedro Antonio Gutiérrez,et al.  Evolutionary product-unit neural networks classifiers , 2008, Neurocomputing.

[29]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[30]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[31]  Pedro Antonio Gutiérrez,et al.  Classification by means of Evolutionary Product-Unit Neural Networks , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[32]  Claudio Moraga,et al.  AD-SVMs: A light extension of SVMs for multicategory classification , 2009, Int. J. Hybrid Intell. Syst..

[33]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[34]  Guido Bugmann,et al.  Normalized Gaussian Radial Basis Function networks , 1998, Neurocomputing.

[35]  Pedro Antonio Gutiérrez,et al.  Evolutionary Product-Unit Neural Networks for Classification , 2006, IDEAL.

[36]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation (3rd Edition) , 2007 .

[37]  R. Lippmann.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[38]  O. J. Dunn.  Multiple Comparisons among Means , 1961 .

[39]  Alexander E. Gegov,et al.  Complexity Management in Fuzzy Systems - A Rule Base Compression Approach , 2007, Studies in Fuzziness and Soft Computing.

[40]  Gerald Sommer,et al.  Evolutionary reinforcement learning of artificial neural networks , 2007, Int. J. Hybrid Intell. Syst..

[41]  CentresMark,et al.  Regularisation in the Selection of Radial Basis Function , 1995 .

[42]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[43]  Hak-Keung Lam,et al.  Tuning of the structure and parameters of a neural network using an improved genetic algorithm , 2003, IEEE Trans. Neural Networks.

[44]  Xin Yao,et al.  Evolving Neural Network Ensembles by Minimization of Mutual Information , 2004, Int. J. Hybrid Intell. Syst..