Local coordinate based graph-regularized NMF for image representation

Abstract Non-negative matrix factorization (NMF) has been a powerful data representation tool which has been widely applied in pattern recognition and computer vision due to its simplicity and effectiveness. However, existing NMF methods suffer from one or both of the following deficiencies: (1) they cannot theoretically guarantee the decomposition results to be sparse, and (2) they completely neglect geometric structure of data, especially when some examples are heavily corrupted. In this paper, we propose a local coordinate based graph regularized NMF method (LCGNMF) to simultaneously overcome both deficiencies. In particular, LCGNMF enforces the learned coefficients to be sparse by incorporating the local coordinate constraint over both factors meanwhile preserving the geometric structure of the data by incorporating graph regularization. To enhance the robustness of NMF, LCGNMF removes the effect of the outliers via the maximum correntropy criterion (MCC). LCGNMF is difficult because the MCC induced objective function is neither quadratic nor convex. We therefore developed a multiplicative update rule to solve LCGNMF and theoretically proved its convergence. Experiments of image clustering on several popular image datasets verify the effectiveness of LCGNMF compared to the representative methods in quantities.

[1]  H. Nkansah.  Least squares optimization with L1-norm regularization , 2017 .

[2]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[3]  Wei Liu,et al.  Nonnegative Local Coordinate Factorization for Image Representation , 2011, IEEE Transactions on Image Processing.

[4]  Zheng Yang,et al.  Locality-Constrained Concept Factorization , 2011, IJCAI.

[5]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[6]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Erkki Oja,et al.  Projective Nonnegative Matrix Factorization for Image Compression and Feature Extraction , 2005, SCIA.

[8]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[10]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.

[11]  Xuelong Li,et al.  General Tensor Discriminant Analysis and Gabor Features for Gait Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  D. Perrett,et al.  Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque. , 1994, Cerebral cortex.

[13]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[15]  Dietrich Lehmann,et al.  Nonsmooth nonnegative matrix factorization (nsNMF) , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[17]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[18]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[19]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[20]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[21]  Xuan Li,et al.  Robust Nonnegative Matrix Factorization via Half-Quadratic Minimization , 2012, 2012 IEEE 12th International Conference on Data Mining.

[22]  Ran He,et al.  Nonnegative sparse coding for discriminative semi-supervised learning , 2011, CVPR 2011.

[23]  Xiaofei He,et al.  Robust non-negative matrix factorization , 2011 .

[24]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[25]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[26]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Sahin Albayrak,et al.  Spectral Analysis of Signed Graphs for Clustering, Prediction and Visualization , 2010, SDM.

[28]  P. Tang,et al.  Subspace Feature Analysis of Local Manifold Learning for Hyperspectral Remote Sensing Images Classification , 2014 .

[29]  M. Tarr,et al.  Visual Object Recognition , 1996, ISTCS.

[30]  S. Palmer.  Hierarchical structure in perceptual representation , 1977, Cognitive Psychology.

[31]  David L. Sheinberg,et al.  Visual object recognition. , 1996, Annual review of neuroscience.

[32]  Zhigang Luo,et al.  Non-Negative Patch Alignment Framework , 2011, IEEE Transactions on Neural Networks.

[33]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[34]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[35]  Zhigang Luo,et al.  Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent , 2011, IEEE Transactions on Image Processing.

[36]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[37]  Dacheng Tao,et al.  Signed Laplacian Embedding for Supervised Dimension Reduction , 2014, AAAI.

[38]  Luo Si,et al.  Non-Negative Matrix Factorization Clustering on Multiple Manifolds , 2010, AAAI.

[39]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Zhigang Luo,et al.  Online Nonnegative Matrix Factorization With Robust Stochastic Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[41]  D. B. Graham,et al.  Characterising Virtual Eigensignatures for General Purpose Face Recognition , 1998 .

[42]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[43]  Xuelong Li,et al.  Local Coordinate Concept Factorization for Image Representation , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[45]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.