Graph regularized multilayer concept factorization for data representation

Previous studies have demonstrated that matrix factorization techniques, such as Nonnegative Matrix Factorization (NMF) and Concept Factorization (CF), have yielded impressive results in image processing and data representation. However, conventional CF and its variants with single layer factorization fail to capture the intrinsic structure of data. In this paper, we propose a novel sequential factorization method, namely Graph regularized Multilayer Concept Factorization (GMCF) for clustering. GMCF is a multi-stage procedure, which decomposes the observation matrix iteratively in a number of layers. In addition, GMCF further incorporates graph Laplacian regularization in each layer to efficiently preserve the manifold structure of data. An efficient iterative updating scheme is developed for optimizing GMCF. The convergence of this algorithm is strictly proved; the computational complexity is detailedly analyzed. Extensive experiments demonstrate that GMCF owns the superiorities in terms of data representation and clustering performance.

[1]  Anuj Srivastava,et al.  Optimal linear representations of images for object recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[3]  Feiping Nie,et al.  Robust Matrix Completion via Joint Schatten p-Norm and lp-Norm Minimization , 2012, 2012 IEEE 12th International Conference on Data Mining.

[4]  Jane You,et al.  Image clustering by hyper-graph regularized non-negative matrix factorization , 2014, Neurocomputing.

[5]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[6]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[7]  Bin Gu,et al.  Incremental Support Vector Learning for Ordinal Regression , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Hui Xiong,et al.  IDR/QR: An Incremental Dimension Reduction Algorithm via QR Decomposition , 2005, IEEE Trans. Knowl. Data Eng..

[10]  Quan-Sen Sun,et al.  A unified multiset canonical correlation analysis framework based on graph embedding for multiple feature extraction , 2015, Neurocomputing.

[11]  Bin Gu,et al.  Incremental learning for ν-Support Vector Regression , 2015, Neural Networks.

[12]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[13]  Jiawei Han,et al.  Locally Consistent Concept Factorization for Document Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[14]  Zheng Yang,et al.  Locality-Constrained Concept Factorization , 2011, IJCAI.

[15]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[16]  Chunxia Zhao,et al.  Multilayer concept factorization for data representation , 2015, 2015 10th International Conference on Computer Science & Education (ICCSE).

[17]  Quan-Sen Sun,et al.  A novel semi-supervised canonical correlation analysis and extensions for multi-view dimensionality reduction , 2014, J. Vis. Commun. Image Represent..

[18]  Andrzej Cichocki,et al.  Multilayer Nonnegative Matrix Factorization Using Projected Gradient Approaches , 2007, Int. J. Neural Syst..

[19]  Richard S. Zemel,et al.  Learning Parts-Based Representations of Data , 2006, J. Mach. Learn. Res..

[20]  Jiawei Han,et al.  Orthogonal Laplacianfaces for Face Recognition , 2006, IEEE Transactions on Image Processing.

[21]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[22]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Xianchao Zhang,et al.  Exploiting constraint inconsistence for dimension selection in subspace clustering: A semi-supervised approach , 2011, Neurocomputing.

[24]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.

[25]  Chris H. Q. Ding,et al.  Simultaneous clustering of multi-type relational data via symmetric nonnegative matrix tri-factorization , 2011, CIKM '11.

[26]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[27]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[28]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[30]  Xingming Sun,et al.  Structural Minimax Probability Machine , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Jing Liu,et al.  Structure preserving non-negative matrix factorization for dimensionality reduction , 2013, Comput. Vis. Image Underst..

[33]  David I. Perrett,et al.  Neurophysiology of shape processing , 1993, Image Vis. Comput..

[34]  Laurens van der Maaten,et al.  Accelerating t-SNE using tree-based algorithms , 2014, J. Mach. Learn. Res..

[35]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[36]  Jian Yang,et al.  Discriminative histograms of local dominant orientation (D-HLDO) for biometric image feature extraction , 2013, Pattern Recognit..

[37]  Xiaofei He,et al.  Using Graph Model for Face Analysis , 2005 .

[38]  Chunxia Zhao,et al.  Constrained Sparse Concept Coding Algorithm with Application to Image Representation , 2014, KSII Trans. Internet Inf. Syst..

[39]  Hassan Ghassemian,et al.  Spectral Unmixing of Hyperspectral Imagery Using Multilayer NMF , 2014, IEEE Geoscience and Remote Sensing Letters.

[40]  Yihong Gong,et al.  Document clustering by concept factorization , 2004, SIGIR '04.

[41]  Xiuwen Liu,et al.  Two-stage optimal component analysis , 2008, Comput. Vis. Image Underst..

[42]  Jim Jing-Yan Wang,et al.  Multiple graph regularized nonnegative matrix factorization , 2013, Pattern Recognit..

[43]  David B. Dunson,et al.  A Bayesian Model for Simultaneous Image Clustering, Annotation and Object Segmentation , 2009, NIPS.

[44]  Fakhri Karray,et al.  An Efficient Concept-Based Mining Model for Enhancing Text Clustering , 2010, IEEE Transactions on Knowledge and Data Engineering.

[45]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[46]  Jin Zhang,et al.  An Effective Way of J Wave Separation Based on Multilayer NMF , 2014, Comput. Math. Methods Medicine.

[47]  Feiping Nie,et al.  Social Trust Prediction Using Rank-k Matrix Recovery , 2013, IJCAI.

[48]  Chunxia Zhao,et al.  Local regularization concept factorization and its semi-supervised extension for image representation , 2015, Neurocomputing.

[49]  Feiping Nie,et al.  Robust Discrete Matrix Completion , 2013, AAAI.

[50]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[51]  A. Cichocki,et al.  Multilayer nonnegative matrix factorisation , 2006 .

[52]  Bin Gu,et al.  A Robust Regularization Path Algorithm for $\nu $ -Support Vector Classification , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[53]  Jun Ye,et al.  Dual-graph regularized concept factorization for clustering , 2014, Neurocomputing.

[54]  Fuming Sun,et al.  Graph regularized and sparse nonnegative matrix factorization with hard constraints for data representation , 2016, Neurocomputing.

[55]  Feiping Nie,et al.  Low-Rank Matrix Recovery via Efficient Schatten p-Norm Minimization , 2012, AAAI.

[56]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.