Cluster-Dependent Feature Selection for the RBF Networks

The paper addresses the problem of a radial basis function network initialization with feature section. Main idea of the proposed approach is to achieve the reduction of data dimensionality through feature selection carried-out independently in each hidden unit of the RBFN. To select features we use the so called cluster-dependent feature selection technique. In this paper three different algorithms for determining unique subset of features for each hidden unit are considered. These are RELIEF, Random Forest and Random-based Ensembles. The processes of feature selection and learning are carried-out by program agents working within a specially designed framework which is also described in the paper. The approach is validated experimentally. Classification results of the RBFN with cluster-dependent feature selection are compared with results obtained using RBFNs implementations with some other types of feature selection methods, over several UCI datasets.

[1]  Novakovic Jasmina,et al.  Wrapper Approach for Feature Selections in RBF Network Classifier , 2011 .

[2]  Juan José Rodríguez Diez,et al.  Rotation-based ensembles of RBF networks , 2006, ESANN.

[3]  Josef A. Nossek,et al.  Construction of interpretable Radial Basis Function classifiers based on the Random Forest kernel , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[4]  Piotr Jedrzejowicz,et al.  e-JABAT - An Implementation of the Web-Based A-Team , 2009, Intelligent Agents in the Evolution of Web and Applications.

[5]  Ireneusz Czarnowski.  Cluster-based instance selection for machine classification , 2010, Knowledge and Information Systems.

[6]  Dapeng Wu,et al.  A RELIEF Based Feature Extraction Algorithm , 2008, SDM.

[7]  Boqin Feng,et al.  Training RBF Neural Network with Hybrid Particle Swarm Optimization , 2006, ISNN.

[8]  Joanna Jedrzejowicz,et al.  Cellular GEP-Induced Classifiers , 2010, ICCCI.

[9]  Peng Zhang,et al.  Kernel-Based Nonparametric Regression Method , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[10]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[11]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[12]  Piotr Jedrzejowicz,et al.  Designing RBF Networks Using the Agent-Based Population Learning Algorithm , 2014, New Generation Computing.

[13]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[14]  James C. Bezdek,et al.  Nearest prototype classifier designs: An experimental study , 2001, Int. J. Intell. Syst..

[15]  Mustapha Lebbah,et al.  Cluster-Dependent Feature Selection through a Weighted Learning Paradigm , 2009, EGC.

[16]  Wengang Zhou,et al.  A novel class dependent feature selection method for cancer biomarker discovery , 2014, Comput. Biol. Medicine.

[17]  Lakhmi C. Jain,et al.  Evolutionary computation in data mining , 2005 .

[18]  Piotr Jedrzejowicz,et al.  An Approach to RBF Initialization with Feature Selection , 2014, IEEE Conf. on Intelligent Systems.