Different metaheuristic strategies to solve the feature selection problem

This paper investigates feature subset selection for dimensionality reduction in machine learning. We provide a brief overview of the feature subset selection techniques that are commonly used in machine learning. Different metaheuristic strategies are proposed to solve the feature selection problem - GRASP, Tabu Search and Memetic Algorithm. These three strategies are compared with a Genetic Algorithm (which is the metaheuristic strategy most frequently used to solve this problem) and with other typical feature selection methods, such as Sequential Forward Floating Selection (SFFS) and Sequential Backward Floating Selection (SBFS). The results show that, in general, GRASP and Tabu Search obtain significantly better results than the other methods.

[1]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Pablo Moscato,et al.  A Gentle Introduction to Memetic Algorithms , 2003, Handbook of Metaheuristics.

[3]  Asoke K. Nandi,et al.  Automatic digital modulation recognition using artificial neural network and genetic algorithm , 2004, Signal Process..

[4]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[5]  Kyung-shik Shin,et al.  A genetic algorithm application in bankruptcy prediction modeling , 2002, Expert Syst. Appl..

[6]  Jerzy W. Bala,et al.  Using Learning to Facilitate the Evolution of Features for Recognizing Visual Concepts , 1996, Evolutionary Computation.

[7]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[8]  Sun I. Kim,et al.  Application of irregular and unbalanced data to predict diabetic nephropathy using visualization and feature selection methods , 2008, Artif. Intell. Medicine.

[9]  Luiz Eduardo Soares de Oliveira,et al.  A Methodology for Feature Selection Using Multiobjective Genetic Algorithms for Handwritten Digit String Recognition , 2003, Int. J. Pattern Recognit. Artif. Intell..

[10]  Pedro Larrañaga,et al.  Feature subset selection by genetic algorithms and estimation of distribution algorithms - A case study in the survival of cirrhotic patients treated with TIPS , 2001, Artif. Intell. Medicine.

[11]  Harald Ganster,et al.  Automated Melanoma Recognition , 2001, IEEE Trans. Medical Imaging.

[12]  M. Tada,et al.  Gene-Expression Profile Changes Correlated with Tumor Progression and Lymph Node Metastasis in Esophageal Cancer , 2004, Clinical Cancer Research.

[13]  Jack Sklansky,et al.  On Automatic Feature Selection , 1988, Int. J. Pattern Recognit. Artif. Intell..

[14]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[15]  Celso C. Ribeiro,et al.  Greedy Randomized Adaptive Search Procedures , 2003, Handbook of Metaheuristics.

[16]  Juha Reunanen,et al.  Overfitting in Making Comparisons Between Variable Selection Methods , 2003, J. Mach. Learn. Res..

[17]  Pedro Larrañaga,et al.  Prototype Selection and Feature Subset Selection by Estimation of Distribution Algorithms. A Case Study in the Survival of Cirrhotic Patients Treated with TIPS , 2001, AIME.

[18]  E. Talbi,et al.  A Genetic Algorithm for Feature Selection in Data-Mining for Genetics , 2001 .

[19]  Chi Hau Chen,et al.  Pattern recognition and signal processing , 1978 .

[20]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[21]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[22]  Pier Luca Lanzi,et al.  Fast feature selection with genetic algorithms: a filter approach , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[23]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[24]  Keinosuke Fukunaga,et al.  Application of the Karhunen-Loève Expansion to Feature Selection and Ordering , 1970, IEEE Trans. Computers.

[25]  Pedro Larrañaga,et al.  Feature Subset Selection by Bayesian network-based optimization , 2000, Artif. Intell..

[26]  T. Poggio,et al.  Multiclass cancer diagnosis using tumor gene expression signatures , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Jihoon Yang,et al.  Prediction of Molecular Bioactivity for Drug Design Using a Decision Tree Algorithm , 2003, Discovery Science.

[28]  Reiner Horst,et al.  A new simplicial cover technique in constrained global optimization , 1992, J. Glob. Optim..

[29]  Ponnuthurai N. Suganthan,et al.  Feature Analysis and Classification of Protein Secondary Structure Data , 2003, ICANN.

[30]  Josef Kittler,et al.  Floating search methods for feature selection with nonmonotonic criterion functions , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[31]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[32]  Panos M. Pardalos,et al.  Handbook of applied optimization , 2002 .

[33]  Ron Kohavi,et al.  Wrappers for performance enhancement and oblivious decision graphs , 1995 .

[34]  Richard J. Enbody,et al.  Further Research on Feature Selection and Classification Using Genetic Algorithms , 1993, ICGA.

[35]  G. Bortolan,et al.  The problem of linguistic approximation in clinical decision making , 1988, Int. J. Approx. Reason..

[36]  John R. Anderson,et al.  Explorations of an Incremental, Bayesian Algorithm for Categorization , 1992, Machine Learning.

[37]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[38]  Jan M. Van Campenhout,et al.  On the Possible Orderings in the Measurement Selection Problem , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[40]  Anil K. Jain,et al.  Image retrieval using color and shape , 1996, Pattern Recognit..

[41]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[42]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[43]  Evangelos Simoudis,et al.  Mining business databases , 1996, CACM.

[44]  Pablo Moscato,et al.  Evolutionary Search of Thresholds for Robust Feature Set Selection: Application to the Analysis of Microarray Data , 2004, EvoWorkshops.

[45]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognit. Lett..