A fast algorithm for joint diagonalization with application to blind source separation – BLISS Technical Report ∗ –

We present a new approach to approximate joint diagonalization of a set of matrices. The main advantages of our method are computational efficiency and generality. The algorithm is based on the Frobeniusnorm formulation of the joint diagonalization problem, and addresses diagonalization with a general, non-orthogonal transformation. The iterative scheme of the algorithm inherits the key ideas from the classical Levenberg-Marquardt algorithm, re-interpreted for a multiplicative iteration update, which is more suitable for the joint diagonalization task than traditional additive iteration updates. The algorithm’s efficiency stems from the sparse approximation of the cost function. Extensive numerical simulations illustrate the performance of the algorithm by a comparison to other leading diagonalization methods and demonstrate its capability to perform blind source separation without requiring the usual pre-whitening of the data.

[1]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[2]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[3]  Schuster,et al.  Separation of a mixture of independent signals using time delayed correlations. , 1994, Physical review letters.

[4]  W. Gautschi,et al.  An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form , 1986 .

[5]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[6]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[7]  A. Bunse-Gerstner,et al.  Numerical Methods for Simultaneous Diagonalization , 1993, SIAM J. Matrix Anal. Appl..

[8]  A. Yeredor.  Blind separation of Gaussian sources via second-order statistics with asymptotically optimal weighting , 2000, IEEE Signal Processing Letters.

[9]  Arie Yeredor,et al.  Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation , 2002, IEEE Trans. Signal Process..

[10]  A. Ziehe,et al.  A LINEAR LEAST-SQUARES ALGORITHM FOR JOINT DIAGONALIZATION , 2003 .

[11]  Noboru Murata,et al.  Multiplicative Nonholonomic/Newton -like Algorithm , 2000, ArXiv.

[12]  M. Joho,et al.  Joint diagonalization of correlation matrices by using Newton methods with application to blind signal separation , 2002, Sensor Array and Multichannel Signal Processing Workshop Proceedings, 2002.

[13]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[14]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[15]  P. Philips,et al.  JADETD : COMBINING HIGHER-ORDER STATISTICS AND TEMPORALINFORMATION FOR BLIND SOURCE SEPARATION ( WITH NOISE ) , 1999 .

[16]  Jean-Francois Cardoso,et al.  Perturbation of joint diagonalizers , 1994 .

[17]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixtures of nonstationary sources , 2001, IEEE Trans. Signal Process..

[18]  Eric Moulines,et al.  Asymptotic performance of second order blind separation , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Jose C. Principe,et al.  Simultaneous Diagonalization in the Frequency Domain (SDIF) for Source Separation , 2000 .

[20]  Dinh Tuan Pham,et al.  Joint Approximate Diagonalization of Positive Definite Hermitian Matrices , 2000, SIAM J. Matrix Anal. Appl..

[21]  Andreas Ziehe,et al.  A Fast Algorithm for Joint Diagonalization with Non-orthogonal Transformations and its Application to Blind Source Separation , 2004, J. Mach. Learn. Res..

[22]  Andreas Ziehe,et al.  TDSEP { an e(cid:14)cient algorithm for blind separation using time structure , 1998 .

[23]  Seungjin Choi,et al.  Blind separation of second-order nonstationary and temporally colored sources , 2001, Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing (Cat. No.01TH8563).