A framework for least squares nonnegative matrix factorizations with Tikhonov regularization

Abstract Nonnegative matrix factorization (NMF) is widely used for dimensionality reduction, clustering and signal unmixing. This paper presents a generic model for least squares NMFs with Tikhonov regularization, which covers many well-known NMF models as well as new models. We also develop a generic updating rule with a simple structure to iteratively solve the optimization problem by constructing a surrogate function, which possesses properties similar to that of the standard NMF. The simulation results demonstrate the power of the framework in which some new algorithms can be derived to provide a performance superior to that of other commonly used methods.

[1]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[2]  Kup-Sze Choi,et al.  Convex nonnegative matrix factorization with manifold regularization , 2015, Neural Networks.

[3]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Tie Zhang,et al.  Generalized EM-Type Reconstruction Algorithms for Emission Tomography , 2012, IEEE Transactions on Medical Imaging.

[5]  Ronald H. Huesman,et al.  Correction for ambiguous solutions in factor analysis using a penalized least squares objective , 2002, IEEE Transactions on Medical Imaging.

[6]  Xinbo Gao,et al.  Semi-Supervised Nonnegative Matrix Factorization via Constraint Propagation , 2016, IEEE Transactions on Cybernetics.

[7]  Sen Jia,et al.  Constrained Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Jing Xiao,et al.  Non-negative matrix factorization as a feature selection tool for maximum margin classifiers , 2011, CVPR 2011.

[9]  Andrzej Cichocki,et al.  Csiszár's Divergences for Non-negative Matrix Factorization: Family of New Algorithms , 2006, ICA.

[10]  I. Jolliffe.  Principal Component Analysis , 2002 .

[11]  Zhigang Luo,et al.  Non-Negative Patch Alignment Framework , 2011, IEEE Transactions on Neural Networks.

[12]  Nancy Bertin,et al.  Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis , 2009, Neural Computation.

[13]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[14]  Dietrich Lehmann,et al.  Nonsmooth nonnegative matrix factorization (nsNMF) , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  W.-T. Zhang,et al.  Iterative Algorithm for Joint Zero Diagonalization With Application in Blind Source Separation , 2011, IEEE Transactions on Neural Networks.

[16]  Lixin Gao,et al.  Scalable Linear Visual Feature Learning via Online Parallel Nonnegative Matrix Factorization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Yuan Gao,et al.  Improving molecular cancer class discovery through sparse non-negative matrix factorization , 2005 .

[18]  Yueyang Teng,et al.  A convergent non-negative deconvolution algorithm with Tikhonov regularization , 2015 .

[19]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[20]  A. Tikhonov,et al.  Numerical Methods for the Solution of Ill-Posed Problems , 1995 .

[21]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[22]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[23]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[24]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[25]  J. Eggert,et al.  Sparse coding and NMF , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[26]  Andrzej Cichocki,et al.  Non-negative matrix factorization with alpha-divergence , 2008, Pattern Recognit. Lett..

[27]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Long Lan,et al.  Soft-constrained nonnegative matrix factorization via normalization , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[29]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[30]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[31]  Daniel D. Lee,et al.  Bayesian regularization and nonnegative deconvolution for room impulse response estimation , 2006, IEEE Transactions on Signal Processing.

[32]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[33]  Inderjit S. Dhillon,et al.  Generalized Nonnegative Matrix Approximations with Bregman Divergences , 2005, NIPS.

[34]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[35]  Yong Xiang,et al.  Adaptive Method for Nonsmooth Nonnegative Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Jianchao Fan,et al.  A Collective Neurodynamic Optimization Approach to Nonnegative Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[37]  D. Hunter,et al.  Optimization Transfer Using Surrogate Objective Functions , 2000 .

[38]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[39]  Seungjin Choi,et al.  Semi-Supervised Nonnegative Matrix Factorization , 2010, IEEE Signal Processing Letters.