Nonnegative tensor factorizations using an alternating direction method

The nonnegative tensor (matrix) factorization finds more and more applications in various disciplines including machine learning, data mining, and blind source separation, etc. In computation, the optimization problem involved is solved by alternatively minimizing one factor while the others are fixed. To solve the subproblem efficiently, we first exploit a variable regularization term which makes the subproblem far from ill-condition. Second, an augmented Lagrangian alternating direction method is employed to solve this convex and well-conditioned regularized subproblem, and two accelerating skills are also implemented. Some preliminary numerical experiments are performed to show the improvements of the new method.

[1]  Jonathan M. Smith,et al.  IDES: An Internet Distance Estimation Service for Large Networks , 2006, IEEE Journal on Selected Areas in Communications.

[2]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[3]  Lars Kai Hansen,et al.  Mining the posterior cingulate: Segregation between memory and pain components , 2005, NeuroImage.

[4]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[5]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[7]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[8]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[9]  Tamara G. Kolda,et al.  Efficient MATLAB Computations with Sparse and Factored Tensors , 2007, SIAM J. Sci. Comput..

[10]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[12]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .

[13]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[14]  Wei Xu,et al.  An operator splitting method for variational inequalities with partially unknown mappings , 2008, Numerische Mathematik.

[15]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[16]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[17]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[18]  Bingsheng He,et al.  Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities , 1998, Oper. Res. Lett..

[19]  Xiao Wang,et al.  Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization , 2011, IEEE Transactions on Network and Service Management.

[20]  Qiang Zhang,et al.  Spectral unmixing using nonnegative tensor factorization , 2007, ACM-SE 45.

[21]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[22]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[23]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[24]  Constantine Kotropoulos,et al.  Non-Negative Tensor Factorization Applied to Music Genre Classification , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[25]  Mikkel N. Schmidt,et al.  Probabilistic non-negative tensor factorization using Markov chain Monte Carlo , 2009, 2009 17th European Signal Processing Conference.

[26]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[27]  B. He,et al.  Alternating Direction Method with Self-Adaptive Penalty Parameters for Monotone Variational Inequalities , 2000 .

[28]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[29]  Haesun Park,et al.  Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons , 2011, SIAM J. Sci. Comput..

[30]  Tamara G. Kolda,et al.  Categories and Subject Descriptors: G.4 [Mathematics of Computing]: Mathematical Software— , 2022 .

[31]  Michael W. Berry,et al.  Email Surveillance Using Non-negative Matrix Factorization , 2005, Comput. Math. Organ. Theory.

[32]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[33]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[34]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .