Ratio sum formula for dimensionality reduction

High-dimensional data analysis often suffers the so-called curse of dimensionality. Therefore, dimensionality reduction is usually carried out on the high-dimensional data before the actual analysis, which is a common and efficient way to eliminate this effect. And the popular trace ratio criterion is an extension of the original linear discriminant analysis (LDA) problem, which involves a search of a transformation matrix W to embed high-dimensional space into a low-dimensional space to achieve dimensionality reduction. However, the trace ratio criterion tends to obtain projection direction with very small variance, which the subset after the projection is diffcult to present the most representative information of the data with maximum efficiency. In this paper, we target on this problem and propose the ratio sum formula for dimensionality reduction. Firstly, we analyze the impact of this trend. Then in order to solve this problem, we propose a new ratio sum formula as well as the solution. In the end, we perform experiments on the Yale-B, ORL, and COIL-20 data sets. The theoretical studies and actual numerical analysis confirm the effectiveness of the proposed method.

[1]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  Feiping Nie,et al.  Fast and Orthogonal Locality Preserving Projections for Dimensionality Reduction. , 2017, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[4]  Xinbo Gao,et al.  Stable Orthogonal Local Discriminant Embedding for Linear Dimensionality Reduction , 2013, IEEE Transactions on Image Processing.

[5]  Hong Qiao,et al.  An improved local tangent space alignment method for manifold learning , 2011, Pattern Recognit. Lett..

[6]  Rong Wang,et al.  Stable and orthogonal local discriminant embedding using trace ratio criterion for dimensionality reduction , 2018, Multimedia Tools and Applications.

[7]  Chunlei Wang,et al.  [Application of improved locally linear embedding algorithm in dimensionality reduction of cancer gene expression data]. , 2014, Sheng wu yi xue gong cheng xue za zhi = Journal of biomedical engineering = Shengwu yixue gongchengxue zazhi.

[8]  Sanmay Das,et al.  Filters, Wrappers and a Boosting-Based Hybrid for Feature Selection , 2001, ICML.

[9]  Feiping Nie,et al.  Trace Ratio Problem Revisited , 2009, IEEE Transactions on Neural Networks.

[10]  Y. Saad,et al.  Numerical Methods for Large Eigenvalue Problems , 2011 .

[11]  Habibollah Haron,et al.  Supervised, Unsupervised, and Semi-Supervised Feature Selection: A Review on Gene Selection , 2016, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[12]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[13]  Gangyao Kuang,et al.  Deep supervised t-SNE for SAR target recognition , 2017, 2017 2nd International Conference on Frontiers of Sensors Technologies (ICFST).

[14]  Bo Du,et al.  Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding , 2015, Pattern Recognit..

[15]  Dong Xu,et al.  Semi-Supervised Dimension Reduction Using Trace Ratio Criterion , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Dong Xu,et al.  Trace Ratio vs. Ratio Trace for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Stefan Gheorghe Pentiuc,et al.  Data Dimensionality Reduction for Data Mining: A Combined Filter-Wrapper Framework , 2014, Int. J. Comput. Commun. Control.

[18]  Rong Wang,et al.  Fast and Orthogonal Locality Preserving Projections for Dimensionality Reduction , 2017, IEEE Transactions on Image Processing.

[19]  Reyer Zwiggelaar,et al.  Multi-criterion mammographic risk analysis supported with multi-label fuzzy-rough feature selection , 2019, Artif. Intell. Medicine.

[20]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[21]  Jing-Yu Yang,et al.  A generalized Foley-Sammon transform based on generalized fisher discriminant criterion and its application to face recognition , 2003, Pattern Recognit. Lett..

[22]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[23]  Pramod K. Varshney,et al.  Dimensionality Reduction for Registration of High-Dimensional Data Sets , 2013, IEEE Transactions on Image Processing.

[24]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Rong Wang,et al.  Scalable Graph-Based Clustering With Nonnegative Relaxation for Large Hyperspectral Image , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Yuan Yan Tang,et al.  Simultaneous Spectral-Spatial Feature Selection and Extraction for Hyperspectral Images , 2019, IEEE Transactions on Cybernetics.

[27]  Rong Wang,et al.  Parameter-Free Weighted Multi-View Projected Clustering with Structured Graph Learning , 2020, IEEE Transactions on Knowledge and Data Engineering.

[28]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[29]  Diego Gutierrez,et al.  A similarity measure for illustration style , 2014, ACM Trans. Graph..

[30]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[31]  Lei Huang,et al.  Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual Search , 2016, IEEE Transactions on Image Processing.

[32]  Wei Liu,et al.  Discriminative Multi-instance Multitask Learning for 3D Action Recognition , 2017, IEEE Transactions on Multimedia.

[33]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[34]  Jiawei Han,et al.  Spectral regression: a unified subspace learning framework for content-based image retrieval , 2007, ACM Multimedia.

[35]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[36]  Feiping Nie,et al.  Efficient semi-supervised feature selection with noise insensitive trace ratio criterion , 2013, Neurocomputing.