A survey of feature selection and feature extraction techniques in machine learning

Dimensionality reduction as a preprocessing step to machine learning is effective in removing irrelevant and redundant data, increasing learning accuracy, and improving result comprehensibility. However, the recent increase of dimensionality of data poses a severe challenge to many existing feature selection and feature extraction methods with respect to efficiency and effectiveness. In the field of machine learning and pattern recognition, dimensionality reduction is important area, where many approaches have been proposed. In this paper, some widely used feature selection and feature extraction techniques have analyzed with the purpose of how effectively these techniques can be used to achieve high performance of learning algorithms that ultimately improves predictive accuracy of classifier. An endeavor to analyze dimensionality reduction techniques briefly with the purpose to investigate strengths and weaknesses of some widely used dimensionality reduction methods is presented.

[1]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[2]  Kandarpa Kumar Sarma,et al.  SVD and PCA features for ANN based detection of diabetes using retinopathy , 2012, CUBE.

[3]  Lalitha Rangarajan,et al.  Bi-level dimensionality reduction methods using feature selection and feature extraction , 2010 .

[4]  Myungsook Klassen,et al.  Nearest Shrunken Centroid as Feature Selection of Microarray Data , 2009, CATA.

[5]  Mykola Pechenizkiy,et al.  Class Noise and Supervised Learning in Medical Domains: The Effect of Feature Extraction , 2006, 19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).

[6]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[7]  P. Soliz,et al.  Independent Component Analysis for Vision-inspired Classification of Retinal Images with Age-related Macular Degeneration , 2008, 2008 IEEE Southwest Symposium on Image Analysis and Interpretation.

[8]  Rangarajan Lalitha,et al.  Multi-Level Dimensionality Reduction Methods Using Feature Selection and Feature Extraction , 2010 .

[9]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[10]  Wilfried N. Gansterer,et al.  On the Relationship Between Feature Selection and Classification Accuracy , 2008, FSDM.

[11]  Aapo Hyvärinen,et al.  Survey on Independent Component Analysis , 1999 .

[12]  Huan Liu,et al.  Searching for Interacting Features , 2007, IJCAI.

[13]  M.M. Van Hulle,et al.  Comparison of Two Feature Extraction Methods Based on Maximization of Mutual Information , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[14]  Shang Lei,et al.  A Feature Selection Method Based on Information Gain and Genetic Algorithm , 2012, 2012 International Conference on Computer Science and Electronics Engineering.

[15]  Chris H. Q. Ding,et al.  Minimum redundancy feature selection from microarray gene expression data , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[16]  Murat Kantarcioglu,et al.  Privacy-Preserving Data Mining Applications in the Malicious Model , 2007 .

[17]  Sri Ramakrishna,et al.  FEATURE SELECTION METHODS AND ALGORITHMS , 2011 .

[18]  Jacqueline J Meulman,et al.  Nonlinear principal components analysis: introduction and application. , 2007, Psychological methods.

[19]  James C. Gee,et al.  An automated drusen detection system for classifying age-related macular degeneration with color fundus photographs , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[20]  Shaohua Zhou,et al.  Probabilistic analysis of kernel principal components : mixture modeling , and classification , 2003 .

[21]  F. Fleuret.  Fast Binary Feature Selection with Conditional Mutual Information , 2004, J. Mach. Learn. Res..

[22]  Hiroshi Motoda,et al.  Feature Selection Extraction and Construction , 2002 .

[23]  R. Tibshirani,et al.  Diagnosis of multiple cancer types by shrunken centroids of gene expression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Marco Vannucci,et al.  Variable Selection and Feature Extraction Through Artificial Intelligence Techniques , 2013 .

[25]  Sylvain Chartier,et al.  An Introduction to Independent Component Analysis: InfoMax and FastICA algorithms , 2010 .

[26]  Kiran Kumar Somasundaram,et al.  An Empirical Study on feature selection for Data Classification , 2012 .

[27]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[28]  Jihoon Yang,et al.  Experimental Comparison of Feature Subset Selection Methods , 2007, Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007).

[29]  L. Ladha,et al.  FEATURE SELECTION METHODS AND ALGORITHMS , 2011 .

[30]  Michael G. Madden,et al.  The effect of principal component analysis on machine learning accuracy with high-dimensional spectral data , 2005, Knowl. Based Syst..