Canonical polyadic decomposition of third-order semi-nonnegative semi-symmetric tensors using LU and QR matrix factorizations

Semi-symmetric three-way arrays are essential tools in blind source separation (BSS) particularly in independent component analysis (ICA). These arrays can be built by resorting to higher order statistics of the data. The canonical polyadic (CP) decomposition of such semi-symmetric three-way arrays allows us to identify the so-called mixing matrix, which contains the information about the intensities of some latent source signals present in the observation channels. In addition, in many applications, such as the magnetic resonance spectroscopy (MRS), the columns of the mixing matrix are viewed as relative concentrations of the spectra of the chemical components. Therefore, the two loading matrices of the three-way array, which are equal to the mixing matrix, are nonnegative. Most existing CP algorithms handle the symmetry and the nonnegativity separately. Up to now, very few of them consider both the semi-nonnegativity and the semi-symmetry structure of the three-way array. Nevertheless, like all the methods based on line search, trust region strategies, and alternating optimization, they appear to be dependent on initialization, requiring in practice a multi-initialization procedure. In order to overcome this drawback, we propose two new methods, called JDLU+ and JDQR+, to solve the problem of CP decomposition of semi-nonnegative semi-symmetric three-way arrays. Firstly, we rewrite the constrained optimization problem as an unconstrained one. In fact, the nonnegativity constraint of the two symmetric modes is ensured by means of a square change of variable. Secondly, a Jacobi-like optimization procedure is adopted because of its good convergence property. More precisely, the two new methods use LU and QR matrix factorizations, respectively, which consist in formulating high-dimensional optimization problems into several sequential polynomial and rational subproblems. By using both LU and QR matrix factorizations, we aim at studying the influence of the used matrix factorization. Numerical experiments on simulated arrays emphasize the advantages of the proposed methods especially the one based on LU factorization, in the presence of high-variance model error and of degeneracies such as bottlenecks. A BSS application on MRS data confirms the validity and improvement of the proposed methods.

[1]  Robert Bregovic,et al.  Multirate Systems and Filter Banks , 2002 .

[2]  Said Moussaoui,et al.  Séparation de sources non-négatives. Application au traitement des signaux de spectroscopie , 2005 .

[3]  Laurent Albera,et al.  Line search and trust region strategies for canonical decomposition of semi-nonnegative semi-symmetric 3rd order tensors , 2014 .

[4]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[5]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[6]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[7]  Florian Roemer,et al.  A semi-algebraic framework for approximate CP decompositions via simultaneous matrix diagonalizations (SECSI) , 2013, Signal Process..

[8]  Pierre Comon,et al.  Computing the polyadic decomposition of nonnegative third order tensors , 2011, Signal Process..

[9]  Lu Wang,et al.  Nonnegative compression for Semi-Nonnegative Independent Component Analysis , 2014, 2014 IEEE 8th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[10]  D. Befroy,et al.  Magnetic Resonance Spectroscopy Studies of Human Metabolism , 2011, Diabetes.

[11]  Arie Yeredor,et al.  On using exact joint diagonalization for noniterative approximate joint diagonalization , 2005, IEEE Signal Processing Letters.

[12]  Xi-Lin Li,et al.  Nonorthogonal Joint Diagonalization Free of Degenerate Solution , 2007, IEEE Transactions on Signal Processing.

[13]  Serge Dégerine,et al.  A Comparative Study of Approximate Joint Diagonalization Algorithms for Blind Source Separation in Presence of Additive Noise , 2007, IEEE Transactions on Signal Processing.

[14]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[15]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[16]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[17]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[18]  M. Wax,et al.  A least-squares approach to joint diagonalization , 1997, IEEE Signal Processing Letters.

[19]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[20]  Lu Wang,et al.  Nonnegative Joint Diagonalization by Congruence Based on LU Matrix Factorization , 2013, IEEE Signal Processing Letters.

[21]  Andreas Ziehe,et al.  A Fast Algorithm for Joint Diagonalization with Non-orthogonal Transformations and its Application to Blind Source Separation , 2004, J. Mach. Learn. Res..

[22]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[23]  Arie Yeredor,et al.  Joint Matrices Decompositions and Blind Source Separation: A survey of methods, identification, and applications , 2014, IEEE Signal Processing Magazine.

[24]  Alle-Jan van der Veen,et al.  Joint diagonalization via subspace fitting techniques , 2001, ICASSP.

[25]  André Lima Férrer de Almeida,et al.  Double Khatri–Rao Space-Time-Frequency Coding Using Semi-Blind PARAFAC Based Receiver , 2013, IEEE Signal Processing Letters.

[26]  Florian Roemer,et al.  Tensor-Based Channel Estimation and Iterative Refinements for Two-Way Relaying With Multiple Antennas and Spatial Reuse , 2010, IEEE Transactions on Signal Processing.

[27]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[28]  Bijan Afsari,et al.  Simple LU and QR Based Non-orthogonal Matrix Joint Diagonalization , 2006, ICA.

[29]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[30]  André Lima Férrer de Almeida,et al.  PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization , 2007, Signal Process..

[31]  Bijan Afsari,et al.  Sensitivity Analysis for the Problem of Matrix Joint Diagonalization , 2008, SIAM J. Matrix Anal. Appl..

[32]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorizations : An algorithmic perspective , 2014, IEEE Signal Processing Magazine.

[33]  El Mostafa Fadaili,et al.  Nonorthogonal Joint Diagonalization/Zero Diagonalization for Source Separation Based on Time-Frequency Distributions , 2007, IEEE Transactions on Signal Processing.

[34]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[35]  Arie Yeredor,et al.  Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation , 2002, IEEE Trans. Signal Process..

[36]  Lotfi Senhadji,et al.  Chapter 18 – Biomedical applications , 2010 .

[37]  Mark D. Plumbley Algorithms for nonnegative independent component analysis , 2003, IEEE Trans. Neural Networks.

[38]  P. Tichavsky,et al.  Fast Approximate Joint Diagonalization Incorporating Weight Matrices , 2009, IEEE Transactions on Signal Processing.

[39]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[40]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .

[41]  Jérôme Pagès,et al.  INDSCAL model: geometrical interpretation and methodology , 2006 .

[42]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[43]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[44]  Wei Xing Zheng,et al.  A Fast Algorithm for Nonunitary Joint Diagonalization and Its Application to Blind Source Separation , 2011, IEEE Transactions on Signal Processing.

[45]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[46]  P. Kroonenberg Applied Multiway Data Analysis , 2008 .

[47]  R. Harshman,et al.  PARAFAC: parallel factor analysis , 1994 .

[48]  Dinh Tuan Pham,et al.  Joint Approximate Diagonalization of Positive Definite Hermitian Matrices , 2000, SIAM J. Matrix Anal. Appl..

[49]  R. Plemmons,et al.  Optimality, computation, and interpretation of nonnegative matrix factorizations , 2004 .

[50]  André Uschmajew,et al.  Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation , 2012, SIAM J. Matrix Anal. Appl..

[51]  André Lima Férrer de Almeida,et al.  Space-Time-Frequency (STF) MIMO Communication Systems With Blind Receiver Based on a Generalized PARATUCK2 Model , 2013, IEEE Transactions on Signal Processing.

[52]  Gilles Chabriel,et al.  A Direct Algorithm for Nonorthogonal Approximate Joint Diagonalization , 2012, IEEE Transactions on Signal Processing.

[53]  Qiang Zhang,et al.  Tensor methods for hyperspectral data analysis: a space object material identification study. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[54]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[55]  Lu Wang,et al.  CP decomposition of semi-nonnegative semi-symmetric tensors based on QR matrix factorization , 2014, 2014 IEEE 8th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[56]  P. Comon,et al.  Blind Identification of Overcomplete MixturEs of sources (BIOME) , 2004 .

[57]  Patrick Dupont,et al.  Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone , 2007, NeuroImage.

[58]  Antoine Souloumiac,et al.  Nonorthogonal Joint Diagonalization by Combining Givens and Hyperbolic Rotations , 2009, IEEE Transactions on Signal Processing.

[59]  L. De Lathauwer,et al.  Algebraic methods after prewhitening , 2010 .

[60]  Michele Conti,et al.  SMA biomedical applications , 2021, Shape Memory Alloy Engineering.

[61]  X. Luciani,et al.  Canonical Polyadic Decomposition based on joint eigenvalue decomposition , 2014 .

[62]  Klaus Obermayer,et al.  Quadratic optimization for simultaneous matrix diagonalization , 2006, IEEE Transactions on Signal Processing.