Enhanced Line Search: A Novel Method to Accelerate PARAFAC

The ALS algorithm, used to fit the Parafac model, sometimes needs a large number of iterations before converging. The slowness in convergence can be due to the large size of the data, or to the presence of degeneracies, etc. Several methods have been proposed to speed up the algorithm, some of which are compression [3], and Line Search [2]. In this paper, after a description of Parafac, we will present a novel method for speeding up the algorithm that shows better results in simulations compared to the existing methods, especially in the case of degeneracy. The paper gives an application of the method to blindly identify the mixing matrix of an Under-Determined Mixture (UDM), but it can be applied to any N-way decomposition problem.

[1]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[2]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[3]  H. Law.  Research methods for multimode data analysis , 1984 .

[4]  R T Ross,et al.  Component resolution using multilinear models. , 1995, Methods in enzymology.

[5]  R. Harshman,et al.  ‘Stretch’ vs ‘slice’ methods for representing three‐way structure via matrix notation , 2002 .

[6]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[7]  Sue Leurgans,et al.  [27] Component resolution using multilinear models , 1995 .

[8]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[9]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[10]  R. Cattell,et al.  FACTOR ROTATION FOR PROPORTIONAL PROFILES: ANALYTICAL SOLUTION AND AN EXAMPLE , 1955 .

[11]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[12]  R. Cattell.  “Parallel proportional profiles” and other principles for determining the choice of factors by rotation , 1944 .

[13]  Rasmus Bro,et al.  The N-way Toolbox for MATLAB , 2000 .

[14]  S. Leurgans,et al.  A Decomposition for Three-Way Arrays , 1993, SIAM J. Matrix Anal. Appl..

[15]  Richard A. Harshman,et al.  Determination and Proof of Minimum Uniqueness Conditions for PARAFAC1 , 1972 .

[16]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[17]  P. Paatero.  The Multilinear Engine—A Table-Driven, Least Squares Program for Solving Multilinear Problems, Including the n-Way Parallel Factor Analysis Model , 1999 .

[18]  Pierre Comon,et al.  Blind channel identification and extraction of more sources than sensors , 1998, Optics & Photonics.

[19]  B. Kowalski,et al.  Tensorial resolution: A direct trilinear decomposition , 1990 .

[20]  Rasmus Bro,et al.  Improving the speed of multiway algorithms: Part II: Compression , 1998 .

[21]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[22]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[23]  A. Franc.  Etude algébrique des multitableaux : apports de l'algèbre tensorielle , 1992 .

[24]  W. Rayens,et al.  Two-factor degeneracies and a stabilization of PARAFAC , 1997 .

[25]  Pierre Comon,et al.  Blind identification of under-determined mixtures based on the characteristic function , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[26]  J. Kruskal.  Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .