A differential evolution approach to dimensionality reduction for classification needs

Abstract The feature selection problem often occurs in pattern recognition and, more specifically, classification. Although these patterns could contain a large number of features, some of them could prove to be irrelevant, redundant or even detrimental to classification accuracy. Thus, it is important to remove these kinds of features, which in turn leads to problem dimensionality reduction and could eventually improve the classification accuracy. In this paper an approach to dimensionality reduction based on differential evolution which represents a wrapper and explores the solution space is presented. The solutions, subsets of the whole feature set, are evaluated using the k-nearest neighbour algorithm. High quality solutions found during execution of the differential evolution fill the archive. A final solution is obtained by conducting k-fold crossvalidation on the archive solutions and selecting the best one. Experimental analysis is conducted on several standard test sets. The classification accuracy of the k-nearest neighbour algorithm using the full feature set and the accuracy of the same algorithm using only the subset provided by the proposed approach and some other optimization algorithms which were used as wrappers are compared. The analysis shows that the proposed approach successfully determines good feature subsets which may increase the classification accuracy.

[1]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[3]  B. Debska,et al.  Application of artificial neural network in food classification. , 2011, Analytica chimica acta.

[4]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[5]  Kazuyuki Murase,et al.  A new hybrid ant colony optimization algorithm for feature selection , 2012, Expert Syst. Appl..

[6]  Mu Zhu,et al.  The K-nearest neighbor algorithm predicted rehabilitation potential better than current Clinical Assessment Protocol. , 2007, Journal of clinical epidemiology.

[7]  Vandana,et al.  Survey of Nearest Neighbor Techniques , 2010, ArXiv.

[8]  Padraig Cunningham,et al.  Overfitting in Wrapper-Based Feature Subset Selection: The Harder You Try the Worse it Gets , 2004, SGAI Conf..

[9]  Hui-Huang Hsu,et al.  Hybrid feature selection by combining filters and wrappers , 2011, Expert Syst. Appl..

[10]  Nickolas Savarimuthu,et al.  SVM ranking with backward search for feature selection in type II diabetes databases , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[11]  Dianhong Wang,et al.  Survey of Improving K-Nearest-Neighbor for Classification , 2007, Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2007).

[12]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[13]  Li-Yeh Chuang,et al.  Improved binary particle swarm optimization using catfish effect for feature selection , 2011, Expert Syst. Appl..

[14]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[15]  Viktor Vegh,et al.  A variant of differential evolution for discrete optimization problems requiring mutually distinct parameters , 2011 .

[16]  Xinjie Yu,et al.  Introduction to evolutionary algorithms , 2010, The 40th International Conference on Computers & Indutrial Engineering.

[17]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[18]  Amit Konar,et al.  Two improved differential evolution schemes for faster global search , 2005, GECCO '05.

[19]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[20]  Abdellah El Moudni,et al.  An autonomous vehicle sequencing problem at intersections: A genetic algorithm approach , 2013, Int. J. Appl. Math. Comput. Sci..

[21]  Bartosz Krawczyk,et al.  Combined classifier based on feature space partitioning , 2012, Int. J. Appl. Math. Comput. Sci..

[22]  Andries Petrus Engelbrecht,et al.  Binary differential evolution strategies , 2007, 2007 IEEE Congress on Evolutionary Computation.

[23]  Kazuyuki Murase,et al.  A new local search based hybrid genetic algorithm for feature selection , 2011, Neurocomputing.

[24]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[25]  Daoliang Li,et al.  An improved genetic algorithm for optimal feature subset selection from multi-character feature set , 2011, Expert Syst. Appl..

[26]  Krzysztof Michalak,et al.  CORRELATION-BASED FEATURE SELECTION STRATEGY IN CLASSIFICATION PROBLEMS , 2006 .

[27]  Daniel Lichtblau Differential Evolution in Discrete Optimization. , 2012 .

[28]  Mário A. T. Figueiredo,et al.  Efficient feature selection filters for high-dimensional data , 2012, Pattern Recognit. Lett..

[29]  Adel Al-Jumaily,et al.  Differential evolution based feature subset selection , 2008, 2008 19th International Conference on Pattern Recognition.

[30]  Adam Krzyzak,et al.  Classification of Breast Cancer Malignancy Using Cytological Images of Fine Needle Aspiration Biopsies , 2008, Int. J. Appl. Math. Comput. Sci..

[31]  Kashif Javed,et al.  Feature Selection Based on Class-Dependent Densities for High-Dimensional Binary Data , 2012, IEEE Transactions on Knowledge and Data Engineering.

[32]  Jian Ma,et al.  Igf-bagging: Information gain based feature selection for bagging , 2011 .

[33]  Keith L. Downing,et al.  Introduction to Evolutionary Algorithms , 2006 .

[34]  Jun Gao,et al.  Rank Aggregation Based Text Feature Selection , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[35]  Arthur C. Sanderson,et al.  Differential evolution for discrete optimization: An experimental study on Combinatorial Auction problems , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[36]  Andries Petrus Engelbrecht,et al.  Binary Differential Evolution , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[37]  Zbigniew Telec,et al.  Nonparametric statistical analysis for multiple comparison of machine learning regression algorithms , 2012, Int. J. Appl. Math. Comput. Sci..

[38]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[39]  Jaroslaw Goclawski,et al.  Neural network segmentation of images from stained cucurbits leaves with colour symptoms of biotic and abiotic stresses , 2012, Int. J. Appl. Math. Comput. Sci..

[40]  Yusof Rubiyah,et al.  A new mutation operation for faster convergence in genetic algorithm feature selection , 2012 .

[41]  Goran Martinovic,et al.  Impact of Double Operators on the Performance of a Genetic Algorithm for Solving the Traveling Salesman Problem , 2011, SEMCCO.

[42]  Maya R. Gupta,et al.  Completely Lazy Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[43]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[44]  Hongwei Li,et al.  A Survey of Distance Metrics for Nominal Attributes , 2010, J. Softw..

[45]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[46]  David G. Stork,et al.  Pattern Classification , 1973 .