Manifold based fisher method for semi-supervised feature selection

Fisher criterion is one of most widely used methods for supervised feature selection. Traditional Fisher based feature selection methods focus on maximizing the distances inter-class and minimizing the distances of samples within the same class. But, they ignore the geometric structure of data in measuring the importance of the features. In this paper, we propose a new semi-supervised feature selection algorithm based on the Fisher criterion and manifold assumption. It redefines the inter-classes scatter matrix by maximizing the margins between different classes. The new inter-class scatter matrix is more robust to data with complex distribution. Also, the proposed algorithm includes a new term which keeps locally reconstruction coefficients of data. We show that the capability of the evaluated feature in keeping the reconstruction coefficients is vital in measuring the importance of this feature, especially in semi-supervised cases. Experiments on benchmark data are conducted to show effectiveness of the proposed method.

[1]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[2]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[3]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[4]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[5]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Xiaogang Wang,et al.  A unified framework for subspace face recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Kilian Stoffel,et al.  Theoretical Comparison between the Gini Index and Information Gain Criteria , 2004, Annals of Mathematics and Artificial Intelligence.

[8]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[9]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[13]  Xiaogang Wang,et al.  Dual-space linear discriminant analysis for face recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[14]  Tao Jiang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[15]  Hujun Bao,et al.  Geodesic Based Semi-supervised Multi-manifold Feature Extraction , 2012, 2012 IEEE 12th International Conference on Data Mining.

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Dong Xu,et al.  Multilinear Discriminant Analysis for Face Recognition , 2007, IEEE Transactions on Image Processing.

[18]  Xiaoqin Zhang,et al.  Isometric Multi-manifold Learning for Feature Extraction , 2012, 2012 IEEE 12th International Conference on Data Mining.

[19]  Jiawei Han,et al.  Generalized Fisher Score for Feature Selection , 2011, UAI.

[20]  Ronald A. Cole,et al.  Spoken Letter Recognition , 1990, HLT.

[21]  Le Song,et al.  Supervised feature selection via dependence estimation , 2007, ICML '07.

[22]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[23]  Sohail Asghar,et al.  A REVIEW OF FEATURE SELECTION TECHNIQUES IN STRUCTURE LEARNING , 2013 .

[24]  Feiping Nie,et al.  Trace Ratio Criterion for Feature Selection , 2008, AAAI.

[25]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[26]  Jian Yang,et al.  KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.