Infinite Feature Selection

Filter-based feature selection has become crucial in many classification settings, especially object recognition, recently faced with feature learning strategies that originate thousands of cues. In this paper, we propose a feature selection method exploiting the convergence properties of power series of matrices, and introducing the concept of infinite feature selection (Inf-FS). Considering a selection of features as a path among feature distributions and letting these paths tend to an infinite number permits the investigation of the importance (relevance and redundancy) of a feature when injected into an arbitrary set of cues. Ranking the importance individuates candidate features, which turn out to be effective from a classification point of view, as proved by a thoroughly experimental section. The Inf-FS has been tested on thirteen diverse benchmarks, comparing against filters, embedded methods, and wrappers, in all the cases we achieve top performances, notably on the classification tasks of PASCAL VOC 2007-2012.

[1]  Steve R. Gunn,et al.  Result Analysis of the NIPS 2003 Feature Selection Challenge , 2004, NIPS.

[2]  S. Ramaswamy,et al.  Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma. , 2002, Cancer research.

[3]  Roberto Guzman-Mart ´ inez,et al.  Feature Selection Stability Assessment Based on the Jensen-Shannon Divergence , 2011 .

[4]  Xiaojin Zhu,et al.  Improving Diversity in Ranking using Absorbing Random Walks , 2007, NAACL.

[5]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[6]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[7]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[8]  Nebojsa Jojic,et al.  Feature Selection Using Counting Grids: Application to Microarray Data , 2012, SSPR/SPR.

[9]  Gizem Karaali.  Book Review: Vector Calculus, Linear Algebra, and Differential Forms: A Unified Approach , 2007 .

[10]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[11]  George H. Weiss,et al.  Book Review: Elements of the Random Walk: An Introduction for Advanced Students and Researchers , 2005 .

[12]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Isabelle Guyon,et al.  Competitive baseline methods set new standards for the NIPS 2003 feature selection benchmark , 2007, Pattern Recognit. Lett..

[14]  Mihir Sen,et al.  Mathematical Methods in Engineering , 2007 .

[15]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[16]  ZissermanAndrew,et al.  The Pascal Visual Object Classes Challenge , 2015 .

[17]  Subhransu Maji,et al.  Fast and Accurate Digit Classification , 2009 .

[18]  Marco Zaffalon,et al.  Robust Feature Selection by Mutual Information Distributions , 2002, UAI.

[19]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Deli Zhao,et al.  Cyclizing Clusters via Zeta Function of a Graph , 2008, NIPS.

[21]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[22]  Yue Han,et al.  Stable Gene Selection from Microarray Data via Sample Weighting , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[23]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[24]  Jiawei Han,et al.  Generalized Fisher Score for Feature Selection , 2011, UAI.

[25]  T. Wieczorek,et al.  Comparison of feature ranking methods based on information entropy , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[26]  Pablo M. Granitto,et al.  SVM Based Feature Selection: Why Are We Using the Dual? , 2010, IBERAMIA.

[27]  Ronald L. Graham,et al.  Concrete Mathematics, a Foundation for Computer Science , 1991, The Mathematical Gazette.

[28]  E Bergshoeff,et al.  Ten Physical Applications of Spectral Zeta Functions , 1996 .

[29]  P. Shum,et al.  Computing matrix inversion with optical networks. , 2013, Optics express.

[30]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[31]  Jianzhong Wang,et al.  An Improved Feature Selection Based on Effective Range for Classification , 2014, TheScientificWorldJournal.

[32]  Todd,et al.  Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning , 2002, Nature Medicine.

[33]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[34]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[35]  P. Bonacich.  Power and Centrality: A Family of Measures , 1987, American Journal of Sociology.

[36]  Pat Langley,et al.  Models of Incremental Concept Formation , 1990, Artif. Intell..

[37]  Roberto Guzmán-Martínez,et al.  Feature Selection Stability Assessment Based on the Jensen-Shannon Divergence , 2011, ECML/PKDD.

[38]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[39]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[40]  Sejong Oh,et al.  A novel divide-and-merge classification for high dimensional datasets , 2013, Comput. Biol. Chem..

[41]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.