Gradient Flow Based Matrix Joint Diagonalization for Independent Component Analysis

Abstract : In this thesis, employing the theory of matrix Lie groups, we develop gradient based flows for the problem of Simultaneous or Joint Diagonalization (JD) of a set of symmetric matrices. This problem has applications in many fields especially in the field of Independent Component Analysis (ICA). We consider both orthogonal and non-orthogonal JD. We view the JD problem as minimization of a common quadric cost function on a matrix group. We derive gradient based flows together with suitable discretizations for minimization of this cost function on the Riemannian manifolds of O(n) and GL(n). We use the developed JD methods to introduce a new class of ICA algorithms that sphere the data, however do not restrict the subsequent search for the un-mixing matrix to orthogonal matrices. These methods provide robust ICA algorithms in Gaussian noise by making effective use of both second and higher order statistics.

[1]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[2]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[3]  R. Howe.  Very Basic Lie Theory , 1983 .

[4]  Gene H. Golub,et al.  Matrix computations , 1983 .

[5]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[6]  P. McCullagh.  Tensor Methods in Statistics , 1987 .

[7]  M. Marcus,et al.  A Survey of Matrix Theory and Matrix Inequalities , 1965 .

[8]  A. Bunse-Gerstner,et al.  Numerical Methods for Simultaneous Diagonalization , 1993, SIAM J. Matrix Anal. Appl..

[9]  J. Marsden,et al.  Introduction to mechanics and symmetry , 1994 .

[10]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[11]  A. Iserles,et al.  Runge-Kutta methods for orthogonal and isospectral flows , 1996 .

[12]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[13]  A. R. Humphries,et al.  Dynamical Systems And Numerical Analysis , 1996 .

[14]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[15]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[16]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[17]  P. Comon,et al.  Blind separation of discrete sources , 1998, IEEE Signal Processing Letters.

[18]  Jerrold E. Marsden,et al.  Introduction to Mechanics and Symmetry: A Basic Exposition of Classical Mechanical Systems , 1999 .

[19]  Liqing Zhang,et al.  Natural gradient algorithm for blind separation of overdetermined mixture with additive noise , 1999, IEEE Signal Processing Letters.

[20]  Dinh Tuan Pham,et al.  Joint Approximate Diagonalization of Positive Definite Hermitian Matrices , 2000, SIAM J. Matrix Anal. Appl..

[21]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixtures of nonstationary sources , 2001, IEEE Trans. Signal Process..

[22]  Eric Moreau,et al.  A generalization of joint-diagonalization criteria for source separation , 2001, IEEE Trans. Signal Process..

[23]  Arie Yeredor,et al.  Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation , 2002, IEEE Trans. Signal Process..

[24]  A. Ziehe,et al.  A LINEAR LEAST-SQUARES ALGORITHM FOR JOINT DIAGONALIZATION , 2003 .