Nonnegative matrix factorization with quadratic programming

Nonnegative matrix factorization (NMF) solves the following problem: find such nonnegative matrices A@?R"+^I^x^J and X@?R"+^J^x^K that Y@?AX, given only Y@?R^I^x^K and the assigned index J (K@?I>=J). Basically, the factorization is achieved by alternating minimization of a given cost function subject to nonnegativity constraints. In the paper, we propose to use quadratic programming (QP) to solve the minimization problems. The Tikhonov regularized squared Euclidean cost function is extended with a logarithmic barrier function (which satisfies nonnegativity constraints), and then using second-order Taylor expansion, a QP problem is formulated. This problem is solved with some trust-region subproblem algorithm. The numerical tests are performed on the blind source separation problems.

[1]  Lucas C. Parra,et al.  Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain , 2004, IEEE Transactions on Medical Imaging.

[2]  Bernt Schiele,et al.  Introducing a weighted non-negative matrix factorization for image classification , 2003, Pattern Recognit. Lett..

[3]  Lucas C. Parra,et al.  Recovery of constituent spectra using non-negative matrix factorization , 2003, SPIE Optics + Photonics.

[4]  J. Nagy,et al.  Enforcing nonnegativity in image reconstruction algorithms , 2000, SPIE Optics + Photonics.

[5]  Andrzej Cichocki,et al.  Non-negative Matrix Factorization with Quasi-Newton Optimization , 2006, ICAISC.

[6]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[7]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[8]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[9]  Jong-Hoon Ahn,et al.  MULTIPLE NONNEGATIVE-MATRIX FACTORIZATION OF DYNAMIC PET IMAGES , 2004 .

[10]  A. R. De Pierro,et al.  On the relation between the ISRA and the EM algorithm for positron emission tomography , 1993, IEEE Trans. Medical Imaging.

[11]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Jordi Vitrià,et al.  Non-negative Matrix Factorization for Face Recognition , 2002, CCIA.

[13]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[14]  Yunde Jia,et al.  Non-negative matrix factorization framework for face recognition , 2005, Int. J. Pattern Recognit. Artif. Intell..

[15]  Andrzej Cichocki,et al.  New Algorithms for Non-Negative Matrix Factorization in Applications to Blind Source Separation , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[16]  Sun Park,et al.  Document Clustering using Non-negative Matrix Factorization and Fuzzy Relationship , 2010 .

[17]  Dietrich Lehmann,et al.  Nonsmooth nonnegative matrix factorization (nsNMF) , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Michael W. Berry,et al.  Text Mining Using Non-Negative Matrix Factorizations , 2004, SDM.

[19]  Andrzej Cichocki,et al.  Multilayer Nonnegative Matrix Factorization Using Projected Gradient Approaches , 2007, Int. J. Neural Syst..

[20]  R. Plemmons,et al.  Optimality, computation, and interpretation of nonnegative matrix factorizations , 2004 .

[21]  K. Lange,et al.  EM reconstruction algorithms for emission and transmission tomography. , 1984, Journal of computer assisted tomography.

[22]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[23]  N. Rao,et al.  Extracting characteristic patterns from genome-wide expression data by non-negative matrix factorization , 2004 .

[24]  Zhaoshui He,et al.  Extended SMART Algorithms for Non-negative Matrix Factorization , 2006, ICAISC.

[25]  Michael W. Spratling.  Learning Image Components for Object Recognition , 2006, J. Mach. Learn. Res..

[26]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[27]  Charles L. Byrne,et al.  Accelerating the EMML algorithm and related iterative algorithms by rescaled block-iterative methods , 1998, IEEE Trans. Image Process..

[28]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[29]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..

[30]  Daniel D. Lee,et al.  APPLICATION OF NON-NEGATIVE MATRIX FACTORIZATION TO DYNAMIC POSITRON EMISSION TOMOGRAPHY , 2001 .

[31]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[32]  Chengyu Liu,et al.  Biclustering of gene expression data by non-smooth non-negative matrix factorization , 2010 .

[33]  P. Sajda,et al.  RECOVERY OF CONSTITUENT SPECTRA IN 3D CHEMICAL SHIFT IMAGING USING NON-NEGATIVE MATRIX FACTORIZATION , 2003 .

[34]  Nanning Zheng,et al.  Non-negative matrix factorization based methods for object recognition , 2004, Pattern Recognit. Lett..

[35]  Andrzej Cichocki,et al.  Csiszár's Divergences for Non-negative Matrix Factorization: Family of New Algorithms , 2006, ICA.

[36]  Morten Mørup,et al.  Nonnegative Matrix Factor 2-D Deconvolution for Blind Single Channel Source Separation , 2006, ICA.

[37]  J. Nagy,et al.  Quasi-Newton approach to nonnegative image restorations , 2000 .

[38]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[39]  A. Cichocki,et al.  Multilayer nonnegative matrix factorisation , 2006 .

[40]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[41]  Jordi Vitrià,et al.  Analyzing non-negative matrix factorization for image classification , 2002, Object recognition supported by user interaction for service robots.

[42]  Lothar Reichel,et al.  TIKHONOV REGULARIZATION WITH NONNEGATIVITY CONSTRAINT , 2004 .

[43]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[44]  Oleg Okun,et al.  Fast Nonnegative Matrix Factorization and Its Application for Protein Fold Recognition , 2006, EURASIP J. Adv. Signal Process..

[45]  T. Adalı,et al.  Non-Negative Matrix Factorization with Orthogonality Constraints for Chemical Agent Detection in Raman Spectra , 2005, 2005 IEEE Workshop on Machine Learning for Signal Processing.

[46]  V. P. Pauca,et al.  Object Characterization from Spectral Data Using Nonnegative Factorization and Information Theory , 2004 .

[47]  L. Lucy.  An iterative technique for the rectification of observed distributions , 1974 .

[48]  Francoise Preteux,et al.  Mathematical Modeling, Estimation, and Imaging , 2000 .

[49]  D. Guillamet,et al.  Classifying Faces with Non-negative Matrix Factorization , 2002 .

[50]  Trond Steihaug,et al.  An interior-point trust-region-based method for large-scale non-negative regularization , 2002 .