Unsupervised feature extraction using a learned graph with clustering structure

Feature extraction, one kind of dimensionality reduction methodology, has aroused considerable research interests during the last few decades. Traditional graph embedding methods construct a fixed graph with original data to fulfill the aim of feature extraction. The lack of the graph learning mechanism leaves room for the improvement of their performances. In this paper, we propose a novel framework, termed as unsupervised feature extraction using a learned graph with clustering structure (LGCS), in which a graph learning mechanism has been presented. To be specific, the proposed LGCS learns both a transformation matrix and an ideal structured graph which incorporates clustering information. To show the effectiveness of the framework, we present a concrete method within our framework, and an iteration algorithm has been designed to solve the corresponding optimizing problem. Promising experimental results on real-world datasets have validated the effectiveness of our proposed algorithm.

[1]  Jing Wang,et al.  Local linear transformation embedding , 2009, Neurocomputing.

[2]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[3]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[4]  Xuelong Li,et al.  Effective Feature Extraction in High-Dimensional Space , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[7]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[8]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[9]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[10]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[11]  Jiawei Han,et al.  Isometric Projection , 2007, AAAI.

[12]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations I. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[14]  WenAn Tan,et al.  Gabor feature-based face recognition using supervised locality preserving projection , 2007, Signal Process..

[15]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[16]  Hong Yan,et al.  Feature Extraction and Uncorrelated Discriminant Analysis for High-Dimensional Data , 2008, IEEE Transactions on Knowledge and Data Engineering.

[17]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[18]  Jeng-Shyang Pan,et al.  Kernel class-wise locality preserving projection , 2008, Inf. Sci..

[19]  F. Westad,et al.  Independent Component Analysis , 2009, Encyclopedia of Biometrics.

[20]  Feiping Nie,et al.  A New Simplex Sparse Learning Model to Measure Data Similarity for Clustering , 2015, IJCAI.

[21]  Joshua B. Tenenbaum,et al.  The Isomap Algorithm and Topological Stability , 2002, Science.

[22]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[23]  B. Mohar.  THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[24]  Yi Wu,et al.  Stable local dimensionality reduction approaches , 2009, Pattern Recognit..

[25]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[26]  Jian Yang,et al.  Globally Maximizing, Locally Minimizing: Unsupervised Discriminant Projection with Applications to Face and Palm Biometrics , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.