A seminorm regularized alternating least squares algorithm for canonical tensor decomposition

Abstract The regularization method could deal with the swamp effect of alternating least squares (ALS) algorithms for tensor decomposition. Usually, the regularization term is a norm of the difference between the solution and the current iterate. In this paper, we show that the norm could be weakened to a seminorm, so the selection of the regularization term could be more flexible. To overcome the swamp effect and avoid the drawback that the Hessian of the subproblem may get close to singular in the iterative process, we propose a seminorm regularized ALS algorithm for solving the canonical tensor decomposition. Moreover, in the new algorithm, we introduce a novel extrapolation in the update of each mode factor which makes an immediate impression on the update of subsequent ones. By assuming the boundness of the infinite sequence of iterates generated by the new algorithm, we establish the global convergence and the (weakly) linear convergence rate of the sequence of iterates Numerical experiments on synthetic and real-world problems illustrate that the new method is efficient and promising.

[1]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[2]  Yi Xu,et al.  Nonnegative Tensor Factorization, Completely Positive Tensors, and a Hierarchical Elimination Algorithm , 2013, SIAM J. Matrix Anal. Appl..

[3]  Lieven De Lathauwer,et al.  A Block Component Model-Based Blind DS-CDMA Receiver , 2008, IEEE Transactions on Signal Processing.

[4]  Lars Kai Hansen,et al.  ERPWAVELAB A toolbox for multi-channel analysis of time–frequency transformed event related potentials , 2007, Journal of Neuroscience Methods.

[5]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[6]  Y. Z. Cao,et al.  A PARAFAC algorithm using penalty diagonalization error (PDE) for three-way data array resolution. , 2000, The Analyst.

[7]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[8]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[9]  L. Qi,et al.  Numerical multilinear algebra and its applications , 2007 .

[10]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[11]  Liqun Qi,et al.  New ALS Methods With Extrapolating Search Directions and Optimal Step Size for Complex-Valued Tensor Decompositions , 2011, IEEE Transactions on Signal Processing.

[12]  P. Paatero.  Construction and analysis of degenerate PARAFAC models , 2000 .

[13]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[14]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[15]  Na Li,et al.  Some Convergence Results on the Regularized Alternating Least-Squares Method for Tensor Decomposition , 2011, 1109.3831.

[16]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[17]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[18]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[19]  André Lima Férrer de Almeida,et al.  PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization , 2007, Signal Process..

[20]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[21]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[22]  Martin J. Mohlenkamp.  Musings on multilinear fitting , 2013 .

[23]  L. Qi.  Eigenvalues and invariants of tensors , 2007 .

[24]  Wenyu Sun,et al.  Positive Semidefinite Generalized Diffusion Tensor Imaging via Quadratic Semidefinite Programming , 2013, SIAM J. Imaging Sci..

[25]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .

[26]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[27]  Fei Wang,et al.  An Eigenvalue Method for Testing Positive Definiteness of a Multivariate Form , 2008, IEEE Transactions on Automatic Control.

[28]  Tamara G. Kolda,et al.  Efficient MATLAB Computations with Sparse and Factored Tensors , 2007, SIAM J. Sci. Comput..

[29]  Hans-Peter Seidel,et al.  Estimating Crossing Fibers: A Tensor Decomposition Approach , 2008, IEEE Transactions on Visualization and Computer Graphics.

[30]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[31]  Marco Donatelli,et al.  Square regularization matrices for large linear discrete ill‐posed problems , 2012, Numer. Linear Algebra Appl..

[32]  Lars Kai Hansen,et al.  Parallel Factor Analysis as an exploratory tool for wavelet transformed event-related EEG , 2006, NeuroImage.

[33]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[34]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[35]  R. Bro.  PARAFAC. Tutorial and applications , 1997 .

[36]  Marco Donatelli,et al.  Fast nonstationary preconditioned iterative methods for ill-posed problems, with application to image deblurring , 2013 .

[37]  Tamara G. Kolda,et al.  Categories and Subject Descriptors: G.4 [Mathematics of Computing]: Mathematical Software— , 2022 .

[38]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..