PARAFAC algorithms for large-scale problems

Parallel factor analysis (PARAFAC) is a tensor (multiway array) factorization method which allows to find hidden factors (component matrices) from a multidimensional data. Most of the existing algorithms for the PARAFAC, especially the alternating least squares (ALS) algorithm need to compute Khatri-Rao products of tall factors and multiplication of large matrices, and due to this require high computational cost and large memory and are not suitable for very large-scale-problems. Hence, PARAFAC for large-scale data tensors is still a challenging problem. In this paper, we propose a new approach based on a modified ALS algorithm which computes Hadamard products, instead Khatri-Rao products, and employs relatively small matrices. The new algorithms are able to process extremely large-scale tensors with billions of entries. Extensive experiments confirm the validity and high performance of the developed algorithm in comparison with other well-known algorithms.

[1]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[2]  Bülent Yener,et al.  Computational analysis of epileptic focus localization , 2006 .

[3]  A. R. De Pierro,et al.  On the relation between the ISRA and the EM algorithm for positron emission tomography , 1993, IEEE Trans. Medical Imaging.

[4]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[5]  Zbynek Koldovský,et al.  Simultaneous search for all modes in multilinear models , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[7]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[8]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[9]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[10]  Berkant Savas,et al.  Handwritten digit classification using higher order singular value decomposition , 2007, Pattern Recognit..

[11]  M. Daube-Witherspoon,et al.  An Iterative Image Space Reconstruction Algorthm Suitable for Volume ECT , 1986, IEEE Transactions on Medical Imaging.

[12]  Jimeng Sun,et al.  Beyond streams and graphs: dynamic tensor analysis , 2006, KDD '06.

[13]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[14]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[15]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[16]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[17]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[18]  Athina P. Petropulu,et al.  PARAFAC-Based Blind Estimation Of Possibly Underdetermined Convolutive MIMO Systems , 2008, IEEE Transactions on Signal Processing.

[19]  H. Lantéri,et al.  COMPARISON BETWEEN ISRA AND RLA ALGORITHMS. USE OF A WIENER FILTER BASED STOPPING CRITERION , 1999 .

[20]  R. Leeb,et al.  BCI Competition 2008 { Graz data set B , 2008 .

[21]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[22]  Jimeng Sun,et al.  Incremental pattern discovery on streams, graphs and tensors , 2008, SKDD.

[23]  Lars Kai Hansen,et al.  Decomposing the time-frequency representation of EEG using non-negative matrix and multi-way factorization , 2006 .

[24]  Rasmus Bro,et al.  Improving the speed of multi-way algorithms:: Part I. Tucker3 , 1998 .

[25]  P. P. B. Eggermont,et al.  Maximum smoothed likelihood density estimation , 1995 .

[26]  L. De Lathauwer,et al.  A tensor-based blind ds-cdma receiver using simultaneous matrix diagonalization , 2007, 2007 IEEE 8th Workshop on Signal Processing Advances in Wireless Communications.

[27]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[28]  S. Goreinov,et al.  A Theory of Pseudoskeleton Approximations , 1997 .