Accelerating Approximate Nonnegative Canonical Polyadic Decomposition using Extrapolation

In this work, we consider the problem of approximate Nonnegative Canonical Polyadic Decomposition (aNCPD) of third-order nonnegative tensors, which boils down to minimizing ‖T − (U ⊗ V ⊗W )Ir‖F over element-wise nonnegative matrices U , V and W . We present an accelerated block coordinate descent algorithm that uses Nesterov-like extrapolation in the form U = U + βk(U − Uk−1). Experimental results showcase the effectiveness of the proposed algorithm with respect to other block-coordinate descent algorithms on illconditioned synthetic data.

[1]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[2]  Hans De Sterck,et al.  Nesterov acceleration of alternating least squares for canonical tensor decomposition: Momentum step size selection and restart mechanisms , 2018, Numer. Linear Algebra Appl..

[3]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[4]  Emmanuel J. Candès,et al.  Adaptive Restart for Accelerated Gradient Schemes , 2012, Foundations of Computational Mathematics.

[5]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[6]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[7]  Pierre Comon,et al.  Uniqueness of Nonnegative Tensor Approximations , 2014, IEEE Transactions on Information Theory.

[8]  Nicolas Gillis,et al.  Accelerating Nonnegative Matrix Factorization Algorithms Using Extrapolation , 2018, Neural Computation.

[9]  Nikos D. Sidiropoulos,et al.  A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization , 2015, IEEE Transactions on Signal Processing.

[10]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[11]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[12]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[13]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[14]  Pierre Comon,et al.  Fast Decomposition of Large Nonnegative Tensors , 2015, IEEE Signal Processing Letters.

[15]  Jeremy E. Cohen.  About Notations in Multiway Array Processing , 2015, ArXiv.