Feature Selection Based on Confidence Machine

In machine learning and pattern recognition, feature selection has been a hot topic in the literature. Unsupervised feature selection is challenging due to the loss of labels which would supply the related information.How to define an appropriate metric is the key for feature selection. We propose a filter method for unsupervised feature selection which is based on the Confidence Machine. Confidence Machine offers an estimation of confidence on a feature'reliability. In this paper, we provide the math model of Confidence Machine in the context of feature selection, which maximizes the relevance and minimizes the redundancy of the selected feature. We compare our method against classic feature selection methods Laplacian Score, Pearson Correlation and Principal Component Analysis on benchmark data sets. The experimental results demonstrate the efficiency and effectiveness of our method.

[1]  Jin Xu,et al.  Dictionary Learning Based on Laplacian Score in Sparse Coding , 2011, MLDM.

[2]  Jane Labadin,et al.  Feature selection based on mutual information , 2015, 2015 9th International Conference on IT in Asia (CITA).

[3]  Haibo He,et al.  Active Dictionary Learning in Sparse Representation Based Classification , 2014, ArXiv.

[4]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[5]  Kjersti Engan,et al.  Multi-frame compression: theory and design , 2000, Signal Process..

[6]  Guang Yang,et al.  Small group human activity recognition , 2012, 2012 19th IEEE International Conference on Image Processing.

[7]  Guang Yang,et al.  Sparse-Representation-Based Classification with Structure-Preserving Dimension Reduction , 2014, Cognitive Computation.

[8]  Jing Wang,et al.  A Hierarchical Neural Network Architecture for Classification , 2012, ISNN.

[9]  Guang Yang,et al.  L 1 Graph Based on Sparse Coding for Feature Selection , 2013, ISNN.

[10]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[11]  Haibo He,et al.  DCPE co-training: Co-training based on diversity of class probability estimation , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[12]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[13]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[14]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[16]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[17]  Tina Eliassi-Rad,et al.  Local Structural Features Threaten Privacy across Social Networks , 2013 .

[18]  Haibo He,et al.  Feature selection based on sparse imputation , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[19]  David G. Lowe,et al.  Multiclass Object Recognition with Sparse, Localized Features , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[21]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.