Ranking to Learn: - Feature Ranking and Selection via Eigenvector Centrality

In an era where accumulating data is easy and storing it inexpensive, feature selection plays a central role in helping to reduce the high-dimensionality of huge amounts of otherwise meaningless data. In this paper, we propose a graph-based method for feature selection that ranks features by identifying the most important ones into arbitrary set of cues. Mapping the problem on an affinity graph-where features are the nodes-the solution is given by assessing the importance of nodes through some indicators of centrality, in particular, the Eigen-vector Centrality (EC). The gist of EC is to estimate the importance of a feature as a function of the importance of its neighbors. Ranking central nodes individuates candidate features, which turn out to be effective from a classification point of view, as proved by a thoroughly experimental section. Our approach has been tested on 7 diverse datasets from recent literature (e.g., biological data and object recognition, among others), and compared against filter, embedded and wrappers methods. The results are remarkable in terms of accuracy, stability and low execution time.

[1]  Kristina Lerman,et al.  Centrality metric for dynamic networks , 2010, MLG '10.

[2]  Xiaolong Deng,et al.  MapReduce based Betweenness Approximation Engineering in Large Scale Graph , 2012 .

[3]  Steve R. Gunn,et al.  Result Analysis of the NIPS 2003 Feature Selection Challenge , 2004, NIPS.

[4]  P. Bonacich.  Power and Centrality: A Family of Measures , 1987, American Journal of Sociology.

[5]  Marco Cristani,et al.  Infinite Feature Selection , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Edwin R. Hancock,et al.  A Graph-Based Approach to Feature Selection , 2011, GbRPR.

[7]  F. R. Pitts.  A GRAPH THEORETIC APPROACH TO HISTORICAL GEOGRAPHY , 1965 .

[8]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[9]  Cristina Granziera,et al.  Infinite feature selection on shore-based biomarkers reveals connectivity modulation after stroke , 2016, 2016 International Workshop on Pattern Recognition in Neuroimaging (PRNI).

[10]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[11]  Isabelle Guyon,et al.  Competitive baseline methods set new standards for the NIPS 2003 feature selection benchmark , 2007, Pattern Recognit. Lett..

[12]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[13]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  W. Garrison.  CONNECTIVITY OF THE INTERSTATE HIGHWAY SYSTEM , 2005 .

[15]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[16]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[17]  Ludmila I. Kuncheva,et al.  A stability index for feature selection , 2007, Artificial Intelligence and Applications.

[18]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[19]  Roberto Guzman-Mart ´ inez,et al.  Feature Selection Stability Assessment Based on the Jensen-Shannon Divergence , 2011 .

[20]  Jimeng Sun,et al.  Centralities in Large Networks: Algorithms and Observations , 2011, SDM.

[21]  T. Wieczorek,et al.  Comparison of feature ranking methods based on information entropy , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[22]  Pablo M. Granitto,et al.  SVM Based Feature Selection: Why Are We Using the Dual? , 2010, IBERAMIA.

[23]  Simone Melzi,et al.  Online Feature Selection for Visual Tracking , 2016, BMVC.

[24]  Roberto Guzmán-Martínez,et al.  Feature Selection Stability Assessment Based on the Jensen-Shannon Divergence , 2011, ECML/PKDD.

[25]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[26]  E. Lander,et al.  Gene expression correlates of clinical prostate cancer behavior. , 2002, Cancer cell.

[27]  D. Bamber.  The area above the ordinal dominance graph and the area below the receiver operating characteristic graph , 1975 .

[28]  Marco Zaffalon,et al.  Robust Feature Selection by Mutual Information Distributions , 2002, UAI.

[29]  S. P. Ghrera,et al.  Time efficient ranking system on map reduce framework , 2015, 2015 Third International Conference on Image Information Processing (ICIIP).

[30]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[31]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[32]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[33]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[35]  Verónica Bolón-Canedo,et al.  Recent advances and emerging challenges of feature selection in the context of big data , 2015, Knowl. Based Syst..

[36]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[37]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[38]  Jiawei Han,et al.  Generalized Fisher Score for Feature Selection , 2011, UAI.