A Combination of Parallel Factor and Independent Component Analysis

Although CPA (canonical/parallel factor analysis) has a unique solution, the actual computation can be made more robust by incorporating extra constraints. In several applications, the factors in one mode are known to be statistically independent. On the other hand, in Independent Component Analysis (ICA), it often makes sense to impose a Khatri-Rao structure on the mixing vectors. In this paper, we propose a new algorithm to impose independence constraints in CPA. Our algorithm implements the algebraic CPA structure and the property of statistical independence simultaneously. Numerical experiments show that our method outperforms in several cases pure CPA, pure ICA, and tensor ICA, a previously proposed method for combining ICA and CPA. We also present a strategy for imposing full or partial symmetry in CPA.

[1]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[2]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[3]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[4]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part I: Lemmas for Partitioned Matrices , 2008, SIAM J. Matrix Anal. Appl..

[5]  N. Sidiropoulos,et al.  Maximum likelihood fitting using ordinary least squares algorithms , 2002 .

[6]  Lieven De Lathauwer,et al.  Blind Deconvolution of DS-CDMA Signals by Means of Decomposition in Rank-$(1,L,L)$ Terms , 2008, IEEE Transactions on Signal Processing.

[7]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[8]  Arogyaswami Paulraj,et al.  An analytical constant modulus algorithm , 1996, IEEE Trans. Signal Process..

[9]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[10]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[11]  Gene H. Golub,et al.  Matrix computations , 1983 .

[12]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[13]  Sabine Van Huffel,et al.  Spatially constrained ICA algorithm with an application in EEG processing , 2011, Signal Process..

[14]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[15]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[16]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[17]  C. F. Beckmann,et al.  Tensorial extensions of independent component analysis for multisubject FMRI analysis , 2005, NeuroImage.

[18]  Lieven De Lathauwer,et al.  Blind Identification of Underdetermined Mixtures by Simultaneous Matrix Diagonalization , 2008, IEEE Transactions on Signal Processing.

[19]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[20]  Stephen M. Smith,et al.  Probabilistic independent component analysis for functional magnetic resonance imaging , 2004, IEEE Transactions on Medical Imaging.

[21]  Sabine Van Huffel,et al.  Algorithm for imposing SOBI-type constraints on the CP model , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[22]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[23]  J. Vandewalle,et al.  An introduction to independent component analysis , 2000 .

[24]  J. Cardoso.  On the Performance of Orthogonal Source Separation Algorithms , 1994 .

[25]  C. L. Nikias,et al.  Signal processing with higher-order spectra , 1993, IEEE Signal Processing Magazine.

[26]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[27]  A. Stegeman.  Comparing Independent Component Analysis and the Parafac model for artificial multi-subject fMRI data , 2007 .

[28]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[29]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[30]  Lieven De Lathauwer,et al.  A Block Component Model-Based Blind DS-CDMA Receiver , 2008, IEEE Transactions on Signal Processing.

[31]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part III: Alternating Least Squares Algorithms , 2008, SIAM J. Matrix Anal. Appl..

[32]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorizations : An algorithmic perspective , 2014, IEEE Signal Processing Magazine.

[33]  Ganesh R. Naik,et al.  Introduction: Independent Component Analysis , 2012 .

[34]  L. Lathauwer,et al.  Dimensionality reduction in higher-order signal processing and rank-(R1,R2,…,RN) reduction in multilinear algebra , 2004 .

[35]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[36]  Arie Yeredor,et al.  Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation , 2002, IEEE Trans. Signal Process..

[37]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[38]  P. Kroonenberg.  Applied Multiway Data Analysis , 2008 .

[39]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[40]  Sabine Van Huffel,et al.  Imposing Independence Constraints in the CP Model , 2007, ICA.

[41]  Joos Vandewalle,et al.  A prewhitening-induced bound on the identification error in independent component analysis , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[42]  Nikos D. Sidiropoulos,et al.  Robust iterative fitting of multilinear models , 2005, IEEE Transactions on Signal Processing.

[43]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[44]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[45]  C. L. Nikias,et al.  Higher-order spectra analysis : a nonlinear signal processing framework , 1993 .

[46]  Lieven De Lathauwer,et al.  An enhanced line search scheme for complex-valued tensor decompositions. Application in DS-CDMA , 2008, Signal Process..

[47]  Lieven De Lathauwer,et al.  Low multilinear rank tensor approximation via semidefinite programming , 2009, 2009 17th European Signal Processing Conference.

[48]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[49]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[50]  J. Cardoso,et al.  On optimal source separation based on second and fourth order cumulants , 1996, Proceedings of 8th Workshop on Statistical Signal and Array Processing.

[51]  Lieven De Lathauwer,et al.  Optimization-Based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-(Lr, Lr, 1) Terms, and a New Generalization , 2013, SIAM J. Optim..

[52]  Jean-Francois Cardoso,et al.  ITERATIVE TECHNIQUES FOR BLIND SOURCE SEPARATION USING ONLY FOURTH-ORDER CUMULANTS , 1992 .

[53]  Arie Yeredor,et al.  Optimization of JADE using a novel optimally weighted joint diagonalization approach , 2004, 2004 12th European Signal Processing Conference.

[54]  P. Paatero.  The Multilinear Engine—A Table-Driven, Least Squares Program for Solving Multilinear Problems, Including the n-Way Parallel Factor Analysis Model , 1999 .

[55]  B. Kowalski,et al.  Tensorial resolution: A direct trilinear decomposition , 1990 .

[56]  Chrysostomos L. Nikias,et al.  Higher-order spectral analysis , 1993, Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Societ.

[57]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[58]  S. Leurgans,et al.  A Decomposition for Three-Way Arrays , 1993, SIAM J. Matrix Anal. Appl..