Trust-Region Strategy with Cauchy Point for Nonnegative Tensor Factorization with Beta-Divergence

Nonnegative tensor factorization is a well-known unsupervised learning method for multi-linear feature extraction from a nonnegatively constrained multi-way array. Many computational strategies have been proposed for updating nonnegative factor matrices in this factorization model but they are mostly restricted to minimization of the objective function expressed by the Euclidean distance. Minimization of other functions, such as the beta-divergence, is more challenging and usually leads to higher complexity. In this study, the trust-region (TR) algorithm is used for minimization of the beta-divergence. We noticed that the Cauchy point strategy in the TR algorithm can be simplified for this function, which is profitable for updating the factors in the discussed model. The experiments show high efficiency of the proposed approach.

[1]  Michael P. Friedlander,et al.  Computing non-negative tensor factorizations , 2008, Optim. Methods Softw..

[2]  Jun Zhou,et al.  Hyperspectral Imagery Denoising via Reweighed Sparse Low-Rank Nonnegative Tensor Factorization , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[3]  Hyunsoo Kim,et al.  Non-negative Tensor Factorization Based on Alternating Large-scale Non-negativity-constrained Least Squares , 2007, 2007 IEEE 7th International Symposium on BioInformatics and BioEngineering.

[4]  Johnathan M. Bardsley.  A NONNEGATIVELY CONSTRAINED TRUST REGION ALGORITHM FOR THE RESTORATION OF IMAGES WITH AN UNKNOWN BLUR , 2005 .

[5]  M. C. Jones,et al.  Robust and efficient estimation by minimising a density power divergence , 1998 .

[6]  Derry Fitzgerald,et al.  Extended Nonnegative Tensor Factorisation Models for Musical Sound Source Separation , 2008, Comput. Intell. Neurosci..

[7]  Ali Taylan Cemgil,et al.  Non-negative tensor factorization models for Bayesian audio processing , 2015, Digit. Signal Process..

[8]  Tamir Hazan,et al.  Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization , 2006, ECCV.

[9]  Sergio Cruces,et al.  Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization , 2011, Entropy.

[10]  Rafal Zdunek.  Trust-Region Algorithm for Nonnegative Matrix Factorization with Alpha- and Beta-divergences , 2012, DAGM/OAGM Symposium.

[11]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with quadratic programming , 2008, Neurocomputing.

[12]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[13]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[14]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[15]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[16]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[17]  Andrzej Cichocki,et al.  Non-Negative Tensor Factorization using Alpha and Beta Divergences , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[18]  Lars Kai Hansen,et al.  Algorithms for Sparse Nonnegative Tucker Decompositions , 2008, Neural Computation.

[19]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[20]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[21]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[22]  Christoph Schnörr,et al.  Controlling Sparseness in Non-negative Tensor Factorization , 2006, ECCV.

[23]  Trond Steihaug,et al.  An interior-point trust-region-based method for large-scale non-negative regularization , 2002 .