Adaptive total-variation for non-negative matrix factorization on manifold

Abstract Non-negative matrix factorization (NMF) has been widely applied in information retrieval and computer vision. However, its performance has been restricted due to its limited tolerance to data noise, as well as its inflexibility in setting regularization parameters. In this paper, we propose a novel sparse matrix factorization method for data representation to solve these problems, termed Adaptive Total-Variation Constrained based Non-Negative Matrix Factorization on Manifold (ATV-NMF). The proposed ATV can adaptively choose the anisotropic smoothing scheme based on the gradient information of data to denoise or preserve feature details by incorporating adaptive total variation into the factorization process. Notably, the manifold graph regularization is also incorporated into NMF, which can discover intrinsic geometrical structure of data to enhance the discriminability. Experimental results demonstrate that the proposed method is very effective for data clustering in comparison to the state-of-the-art algorithms on several standard benchmarks.

[1]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Hongkai Zhao,et al.  Multilevel bioluminescence tomography based on radiative transfer equation part 2: total variation and l1 data fidelity. , 2010, Optics express.

[3]  Zhi-Quan Luo,et al.  A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing , 2015, IEEE Signal Processing Magazine.

[4]  Stacey Levine,et al.  Image Restoration via Nonstandard Diffusion , 2004 .

[5]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[6]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[7]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[8]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[9]  Jiawei Han,et al.  Isometric Projection , 2007, AAAI.

[10]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.

[11]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[12]  Gautham J. Mysore,et al.  Stopping Criteria for Non-Negative Matrix Factorization Based Supervised and Semi-Supervised Source Separation , 2014, IEEE Signal Processing Letters.

[13]  Yuan Yan Tang,et al.  Topology Preserving Non-negative Matrix Factorization for Face Recognition , 2008, IEEE Transactions on Image Processing.

[14]  Franz Pernkopf,et al.  Sparse nonnegative matrix factorization with ℓ0-constraints , 2012, Neurocomputing.

[15]  Yu-Jin Zhang,et al.  Nonnegative Matrix Factorization: A Comprehensive Review , 2013, IEEE Transactions on Knowledge and Data Engineering.

[16]  Licheng Jiao,et al.  An efficient matrix factorization based low-rank representation for subspace clustering , 2013, Pattern Recognit..

[17]  Michael Lindenbaum,et al.  Nonnegative Matrix Factorization with Earth Mover's Distance Metric for Image Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[19]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[20]  Yuan Yan Tang,et al.  Total variation norm-based nonnegative matrix factorization for identifying discriminant representation of image patterns , 2008, Neurocomputing.

[21]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[23]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Zhigang Luo,et al.  Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent , 2011, IEEE Transactions on Image Processing.

[25]  Lucas C. Parra,et al.  Recovery of constituent spectra using non-negative matrix factorization , 2003, SPIE Optics + Photonics.

[26]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[27]  Jie Tian,et al.  Total variation regularization for bioluminescence tomography with the split Bregman method. , 2012, Applied optics.

[28]  Zhang Yi,et al.  Document clustering using locality preserving indexing and support vector machines , 2008, Soft Comput..

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  Fei Wang,et al.  Fast affinity propagation clustering: A multilevel approach , 2012, Pattern Recognit..

[31]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[32]  Hongwei Liu,et al.  Nonnegative matrix factorization with bounded total variational regularization for face recognition , 2010, Pattern Recognit. Lett..

[33]  Li Jia,et al.  Nonnegative Matrix Factorization With Regularizations , 2014, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[34]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .