Semi-nonnegative joint diagonalization by congruence and semi-nonnegative ICA

In this paper, we focus on the Joint Diagonalization by Congruence (JDC) decomposition of a set of matrices, while imposing nonnegative constraints on the joint diagonalizer. The latter will be referred to the semi-nonnegative JDC fitting problem. This problem appears in semi-nonnegative Independent Component Analysis (ICA), say ICA involving nonnegative static mixtures, such as those encountered for instance in image processing and in magnetic resonance spectroscopy. In order to achieve the semi-nonnegative JDC decomposition, we propose two novel algorithms called ELS-ALSexp and CGexp, which optimize an unconstrained problem obtained by means of an exponential change of variable. The proposed methods are based on the line search strategy for which an analytic global plane search procedure has been considered. All derivatives have been jointly calculated in matrix form using the algebraic basis for matrix calculus and product operator properties. Our algorithms have been tested on synthetic arrays and the semi-nonnegative ICA problem is illustrated through simulations in magnetic resonance spectroscopy and in image processing. The numerical results show the benefit of using a priori information, such as nonnegativity.

[1]  Laurent Albera,et al.  Line search and trust region strategies for canonical decomposition of semi-nonnegative semi-symmetric 3rd order tensors , 2014 .

[2]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[3]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[4]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[5]  Are Hjørungnes,et al.  Complex-Valued Matrix Differentiation: Techniques and Key Results , 2007, IEEE Transactions on Signal Processing.

[6]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[7]  P. Comon.  Independent Component Analysis , 1992 .

[8]  Pierre Comon,et al.  Computing the polyadic decomposition of nonnegative third order tensors , 2011, Signal Process..

[9]  Dinh Tuan Pham,et al.  Joint Approximate Diagonalization of Positive Definite Hermitian Matrices , 2000, SIAM J. Matrix Anal. Appl..

[10]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[11]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[12]  Chang-Yeong Kim,et al.  Multiple object decomposition based on independent component analysis of multi-energy x-ray projections , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[13]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[14]  Jean-Jacques Bellanger,et al.  Spatial Characterization and Classification of Rectal Bleeding in Prostate Cancer Radiotherapy with a Voxel-Based Principal Components Analysis Model for 3D Dose Distribution , 2011, Prostate Cancer Imaging.

[15]  A. Franc.  Etude algébrique des multitableaux : apports de l'algèbre tensorielle , 1992 .

[16]  P. Weil,et al.  Simulation of excess phase in bipolar transistors , 1978 .

[17]  Joe Brewer,et al.  Kronecker products and matrix calculus in system theory , 1978 .

[18]  Lotfi Senhadji,et al.  Semi-nonnegative Independent Component Analysis: The (3, 4)-SENICAexp Method , 2010, LVA/ICA.

[19]  D. Brie,et al.  Separation of Non-Negative Mixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sampling , 2006, IEEE Transactions on Signal Processing.

[20]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[21]  Henk A. L. Kiers,et al.  Some clarifications of the CANDECOMP algorithm applied to INDSCAL , 1991 .

[22]  P. Comon,et al.  Ica: a potential tool for bci systems , 2008, IEEE Signal Processing Magazine.

[23]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[24]  Liqun Qi,et al.  New ALS Methods With Extrapolating Search Directions and Optimal Step Size for Complex-Valued Tensor Decompositions , 2011, IEEE Transactions on Signal Processing.

[25]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[26]  Jérôme Pagès,et al.  INDSCAL model: geometrical interpretation and methodology , 2006 .

[27]  Yoshio Takane,et al.  Applications of Multidimensional Scaling in Psychometrics , 2005 .

[28]  Theodoros N. Arvanitis,et al.  A comparative study of feature extraction and blind source separation of independent component analysis (ICA) on childhood brain tumour 1H magnetic resonance spectra , 2009, NMR in biomedicine.

[29]  Bijan Afsari,et al.  Sensitivity Analysis for the Problem of Matrix Joint Diagonalization , 2008, SIAM J. Matrix Anal. Appl..

[30]  Arie Yeredor,et al.  Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation , 2002, IEEE Trans. Signal Process..

[31]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[32]  Lotfi Senhadji,et al.  Canonical decomposition of semi-symmetric semi-nonnegative three-way arrays , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[33]  P. Comon,et al.  Blind Identification of Overcomplete MixturEs of sources (BIOME) , 2004 .

[34]  Christophe Ladroue,et al.  Independent component analysis for automated decomposition of in vivo magnetic resonance spectra , 2003, Magnetic resonance in medicine.

[35]  Hadi Parastar,et al.  Is independent component analysis appropriate for multivariate resolution in analytical chemistry , 2012 .

[36]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[37]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[38]  Eric Moreau,et al.  A generalization of joint-diagonalization criteria for source separation , 2001, IEEE Trans. Signal Process..

[39]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[40]  Nobuhiko Kitawaki,et al.  Combined approach of array processing and independent component analysis for blind separation of acoustic signals , 2003, IEEE Trans. Speech Audio Process..

[41]  J. Pulkkinen,et al.  Independent component analysis to proton spectroscopic imaging data of human brain tumours. , 2005, European journal of radiology.

[42]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[43]  Laurent Albera,et al.  Multi-way space-time-wave-vector analysis for EEG source separation , 2012, Signal Process..

[44]  Yoshio Takane,et al.  11 Applications of Multidimensional Scaling in Psychometrics , 2006 .