Trust-Region Algorithm for Nonnegative Matrix Factorization with Alpha- and Beta-divergences

Nonnegative Matrix Factorization (NMF) is a dimensionality reduction method for representing nonnegative data in a low-dimensional nonnegative space. NMF problems are usually solved with an alternating minimization of a given objective function, using nonnegativity constrained optimization algorithms. This paper is concerned with the projected trust-region algorithm that is adapted to minimize a family of divergences or statistical distances, such as α- or β-divergences that are efficient for solving NMF problems. Using the Cauchy point estimate for the quadratic approximation model, a radius of the trust-region can be estimated efficiently for a symmetric and block-diagonal structure of the corresponding Hessian matrices. The experiments demonstrate a high efficiency of the proposed approach.

[1]  Bernt Schiele,et al.  Introducing a weighted non-negative matrix factorization for image classification , 2003, Pattern Recognit. Lett..

[2]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[3]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[4]  Christof Koch,et al.  Detecting and Estimating Signals over Noisy and Unreliable Synapses: Information-Theoretic Analysis , 2001, Neural Computation.

[5]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[6]  Jérôme Idier,et al.  Algorithms for nonnegative matrix factorization with the beta-divergence , 2010, ArXiv.

[7]  Nancy Bertin,et al.  Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis , 2009, Neural Computation.

[8]  Jérôme Idier,et al.  Algorithms for Nonnegative Matrix Factorization with the β-Divergence , 2010, Neural Computation.

[9]  Leszek Wojnar,et al.  Image Analysis , 1998 .

[10]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[11]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Wen Gao,et al.  Unsupervised Texture Classification: Automatically Discover and Classify Texture Patterns , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[13]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[14]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with quadratic programming , 2008, Neurocomputing.

[15]  Thomas Mauthner,et al.  Instant Action Recognition , 2009, SCIA.

[16]  Johnathan M. Bardsley.  A NONNEGATIVELY CONSTRAINED TRUST REGION ALGORITHM FOR THE RESTORATION OF IMAGES WITH AN UNKNOWN BLUR , 2005 .

[17]  Feng Qianjin,et al.  Projected gradient methods for Non-negative Matrix Factorization based relevance feedback algorithm in medical image retrieval , 2011 .

[18]  Christoph Schnörr,et al.  Learning Sparse Representations by Non-Negative Matrix Factorization and Sequential Cone Programming , 2006, J. Mach. Learn. Res..

[19]  Thomas P. Minka,et al.  Divergence measures and message passing , 2005 .

[20]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with constrained second-order optimization , 2007, Signal Process..

[21]  Chong-Yung Chi,et al.  Nonnegative Least-Correlated Component Analysis for Separation of Dependent Sources by Volume Maximization , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Trond Steihaug,et al.  An interior-point trust-region-based method for large-scale non-negative regularization , 2002 .