Classification by evolutionary generalised radial basis functions

This paper proposes a Neural Network model using Generalised kernel functions for the hidden layer of a feed forward network. These functions are Generalised Radial Basis Functions (GRBF), and the architecture, weights and node topology are learned through an evolutionary algorithm. The proposed model is compared with the corresponding standard hidden-node models: Product Unit (PU) neural networks, Multilayer Perceptrons (MLP) with Sigmoidal Units (SUs) and the RBF neural networks. The proposed methodology is tested using twelve benchmark classification datasets from well-known machine learning problems. GRBFs are found to perform better than other standard basis functions at the classification task.

[1]  Chee Peng Lim,et al.  A hybrid SOM-kMER model for data visualization and classification , 2005, Int. J. Hybrid Intell. Syst..

[2]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[3]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[4]  R. Lippmann.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[5]  Chris Bishop,et al.  Improving the Generalization Properties of Radial Basis Function Neural Networks , 1991, Neural Computation.

[6]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[7]  De-Shuang Huang,et al.  A mended hybrid learning algorithm for radial basis function neural networks to improve generalization capability , 2007 .

[8]  David E. Rumelhart,et al.  Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks , 1989, Neural Computation.

[9]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[11]  John E. Moody,et al.  Fast adaptive k-means clustering: some empirical results , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[12]  Pedro Antonio Gutiérrez,et al.  Classification by means of Evolutionary Product-Unit Neural Networks , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[13]  L.N. de Castro,et al.  An evolutionary clustering technique with local search to design RBF neural network classifiers , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[14]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[15]  Carl G. Looney,et al.  Radial basis functional link nets and fuzzy reasoning , 2002, Neurocomputing.

[16]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[17]  Khalid Ali,et al.  Proof , 2006, BMJ : British Medical Journal.

[18]  Antonio J. Rivera,et al.  CO2RBFN for short-term forecasting of the extra virgin olive oil price in the Spanish market , 2010, Int. J. Hybrid Intell. Syst..

[19]  Witold Pedrycz,et al.  Improving RBF networks performance in regression tasks by means of a supervised fuzzy clustering , 2006, Neurocomputing.

[20]  Claudio Moraga,et al.  AD-SVMs: A light extension of SVMs for multicategory classification , 2009, Int. J. Hybrid Intell. Syst..

[21]  F. J. Martı́nez-Estudilloa,et al.  Evolutionary product-unit neural networks classifiers , 2008 .

[22]  O. J. Dunn.  Multiple Comparisons among Means , 1961 .

[23]  Alexander E. Gegov,et al.  Complexity Management in Fuzzy Systems - A Rule Base Compression Approach , 2007, Studies in Fuzziness and Soft Computing.

[24]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[25]  Kwang Bo Cho,et al.  Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction , 1996, Fuzzy Sets Syst..

[26]  Lei Xu,et al.  RBF nets, mixture experts, and Bayesian Ying-Yang learning , 1998, Neurocomputing.

[27]  Boleslaw K. Szymanski,et al.  Introduction to Scientific Data Mining: Direct Kernel Methods and Applications , 2004, Computationally Intelligent Hybrid Systems.

[28]  Guido Bugmann,et al.  Normalized Gaussian Radial Basis Function networks , 1998, Neurocomputing.

[29]  Michel Verleysen,et al.  On the Effects of Dimensionality on Data Analysis with Neural Networks , 2009, IWANN.

[30]  Ignacio Rojas,et al.  Statistical analysis of the parameters of a neuro-genetic algorithm , 2002, IEEE Trans. Neural Networks.

[31]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[32]  James D. Keeler,et al.  Layered Neural Networks with Gaussian Hidden Units as Universal Approximations , 1990, Neural Computation.

[33]  Pedro Antonio Gutiérrez,et al.  Combined projection and kernel basis functions for classification in evolutionary neural networks , 2009, Neurocomputing.

[34]  Damien Franois.  High-dimensional Data Analysis: From Optimal Metrics to Feature Selection , 2008 .

[35]  H. Keselman,et al.  Multiple Comparison Procedures , 2005 .

[36]  Gerald Sommer,et al.  Evolutionary reinforcement learning of artificial neural networks , 2007, Int. J. Hybrid Intell. Syst..

[37]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[38]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[39]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[40]  Hak-Keung Lam,et al.  Tuning of the structure and parameters of a neural network using an improved genetic algorithm , 2003, IEEE Trans. Neural Networks.

[41]  Xin Yao,et al.  Evolving Neural Network Ensembles by Minimization of Mutual Information , 2004, Int. J. Hybrid Intell. Syst..

[42]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .