Unsupervised Feature Selection With Ordinal Preserving Self-Representation

Unsupervised feature selection is designed to select an optimal feature subset without any label information from high-dimensional data, which is implemented by eliminating the irrelevant and redundant features and has been attracted widespread attention in recent years. Specifically, the obtained low-dimensional representation is interpretable that is useful to machine learning applications. In this paper, we propose a novel unsupervised feature selection algorithm, namely ordinal preserving self-representation (OPSR) for image classification and clustering. First, each feature in high-dimensional data is represented by the linear combination of other features. Then, the topology information is introduced into the objective function for utilizing the ordinal locality of high-dimensional data adequately. At last, an efficient iteratively update algorithm is designed to solve the proposed OPSR, and its convergence is proved in detail. Extensive experimental results on six benchmark databases demonstrate that the effectiveness of the OPSR and its superiority also is verified by comparing with some state-of-the-art feature selection algorithms.

[1]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Jianzhong Wang,et al.  Adaptive multiple graph regularized semi-supervised extreme learning machine , 2018, Soft Comput..

[3]  Tieniu Tan,et al.  Feature Selection Based on Structured Sparsity: A Comprehensive Study , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[5]  Ran He,et al.  Unsupervised feature selection with ordinal locality , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[6]  Jianzhong Wang,et al.  Label propagation based semi-supervised non-negative matrix factorization for feature extraction , 2015, Neurocomputing.

[7]  Xiangtao Zheng,et al.  Discovering Diverse Subset for Unsupervised Hyperspectral Band Selection , 2017, IEEE Transactions on Image Processing.

[8]  Pichao Wang,et al.  Robust unsupervised feature selection via dual self-representation and manifold regularization , 2018, Knowl. Based Syst..

[9]  Xiaofeng Zhu,et al.  Unsupervised feature selection via local structure learning and sparse learning , 2017, Multimedia Tools and Applications.

[10]  ChengXiang Zhai,et al.  Robust Unsupervised Feature Selection , 2013, IJCAI.

[11]  Feiping Nie,et al.  Self-Weighted Supervised Discriminative Feature Selection , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[13]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[14]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[15]  Gavin Brown,et al.  Simple strategies for semi-supervised feature selection , 2017, Machine Learning.

[16]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[17]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[18]  Aleksandra Pizurica,et al.  Semisupervised Local Discriminant Analysis for Feature Extraction in Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Jie Tian,et al.  Robust graph regularized unsupervised feature selection , 2018, Expert Syst. Appl..

[20]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[21]  Yang Wang,et al.  Locality constrained Graph Optimization for Dimensionality Reduction , 2017, Neurocomputing.

[22]  Qing Xie,et al.  Self-representation dimensionality reduction for multi-model classification , 2017, Neurocomputing.

[23]  Ling Shao,et al.  Marginal Representation Learning With Graph Structure Self-Adaptation. , 2018, IEEE transactions on neural networks and learning systems.

[24]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[25]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Chengdong Wu,et al.  Double regularized matrix factorization for image classification and clustering , 2018, EURASIP J. Image Video Process..

[27]  Jianzhong Wang,et al.  Unsupervised feature selection by regularized matrix factorization , 2018, Neurocomputing.

[28]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[29]  Chengqi Zhang,et al.  Convex Sparse PCA for Unsupervised Feature Learning , 2014, ACM Trans. Knowl. Discov. Data.

[30]  Kewei Cheng,et al.  Feature Selection , 2016, ACM Comput. Surv..

[31]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Shulin Wang,et al.  Feature selection in machine learning: A new perspective , 2018, Neurocomputing.

[33]  Qinghua Zheng,et al.  An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition , 2018, IEEE Transactions on Cybernetics.

[34]  Qinghua Zheng,et al.  Adaptive Unsupervised Feature Selection With Structure Regularization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[36]  Shichao Zhang,et al.  Unsupervised feature selection by combining subspace learning with feature self-representation , 2018, Pattern Recognit. Lett..

[37]  Xiaojun Chen,et al.  Local Adaptive Projection Framework for Feature Selection of Labeled and Unlabeled Data , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[39]  Mohammad Ali Zare Chahooki,et al.  A Survey on semi-supervised feature selection methods , 2017, Pattern Recognit..

[40]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[41]  Qinghua Hu,et al.  Subspace clustering guided unsupervised feature selection , 2017, Pattern Recognit..

[42]  Guoliang Luo,et al.  Structure Preserving Non-negative Feature Self-Representation for Unsupervised Feature Selection , 2017, IEEE Access.

[43]  Chao Bi,et al.  Semi-supervised local ridge regression for local matching based face recognition , 2015, Neurocomputing.

[44]  Witold Pedrycz,et al.  Subspace learning for unsupervised feature selection via matrix factorization , 2015, Pattern Recognit..

[45]  Ronghua Shang,et al.  Non-Negative Spectral Learning and Sparse Regression-Based Dual-Graph Regularized Feature Selection , 2018, IEEE Transactions on Cybernetics.

[46]  Chao Bi,et al.  Inner Product Regularized Nonnegative Self Representation for Image Classification and Clustering , 2017, IEEE Access.

[47]  Feiping Nie,et al.  Semi-supervised Feature Selection via Rescaled Linear Regression , 2017, IJCAI.

[48]  Nicu Sebe,et al.  Optimized Graph Learning Using Partial Tags and Multiple Features for Image and Video Annotation , 2016, IEEE Transactions on Image Processing.