Second-Order Multidimensional ICA: Performance Analysis

Independent component analysis (ICA) and blind source separation (BSS) deal with extracting a number of mutually independent elements from a set of observed linear mixtures. Motivated by various applications, this paper considers a more general and more flexible model: the sources can be partitioned into groups exhibiting dependence within a given group but independence between two different groups. We argue that this is tantamount to considering multidimensional components as opposed to the standard ICA case which is restricted to one-dimensional components. The core of the paper is devoted to the statistical analysis of the blind separation of multidimensional components based on second-order statistics, in a piecewise-stationary model. We develop the likelihood and the associated estimating equations for the Gaussian case. We obtain closed-form expressions for the Fisher information matrix and the Cramér-Rao bound of the de-mixing parameters, as well as the mean-square error (MSE) of the component estimates. The derived MSE is valid also for non-Gaussian data. Our analysis is verified through numerical experiments, and its performance is compared to classical ICA in various dependence scenarios, quantifying the gain in the accuracy of component recovery in presence of multidimensional components.

[1]  Christian Jutten,et al.  Fisher Information in Source Separation Problems , 2004, ICA.

[2]  Harold W. Gutch,et al.  Independent Subspace Analysis Is Unique, Given Irreducibility , 2007, ICA.

[3]  Ehud Weinstein,et al.  Maximum likelihood and lower bounds in system identification with non-Gaussian inputs , 1994, IEEE Trans. Inf. Theory.

[4]  Seungjin Choi,et al.  Relative Gradient Learning for Independent Subspace Analysis , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[5]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixtures of nonstationary sources , 2001, IEEE Trans. Signal Process..

[6]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[7]  S. Baillet,et al.  Automated interictal spike detection and source localization in magnetoencephalography using independent components analysis and spatio-temporal clustering , 2004, Clinical Neurophysiology.

[8]  Hagit Messer,et al.  ICA of correlated sources mismodeled as uncorrelated: performance analysis , 2009, 2009 IEEE/SP 15th Workshop on Statistical Signal Processing.

[9]  Visa Koivunen,et al.  Compact CramÉr–Rao Bound Expression for Independent Component Analysis , 2008, IEEE Transactions on Signal Processing.

[10]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[11]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[12]  D. T. Pham,et al.  Blind Separation of Cyclostationary Sources Using Joint Block Approximate Diagonalization , 2007, ICA.

[13]  Alexander Graham,et al.  Kronecker Products and Matrix Calculus: With Applications , 1981 .

[14]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[15]  Joos Vandewalle,et al.  A prewhitening-induced bound on the identification error in independent component analysis , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[16]  E. Oja,et al.  Performance Analysis of the FastICA Algorithm and Cramér – Rao Bounds for Linear Independent Component Analysis , 2010 .

[17]  Anna Tonazzini,et al.  Separation of Correlated Astrophysical Sources Using Multiple-Lag Data Covariance Matrices , 2005, EURASIP J. Adv. Signal Process..

[18]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[19]  Lang Tong,et al.  Indeterminacy and identifiability of blind identification , 1991 .

[20]  Barnabás Póczos,et al.  Undercomplete Blind Subspace Deconvolution , 2007, J. Mach. Learn. Res..

[21]  Hao Shen,et al.  Block Jacobi-Type Methods for Log-Likelihood based Linear Independent Subspace Analysis , 2007, 2007 IEEE Workshop on Machine Learning for Signal Processing.

[22]  Fabian J. Theis,et al.  Towards a general independent subspace analysis , 2006, NIPS.

[23]  Arie Yeredor,et al.  Cramér–Rao-Induced Bound for Blind Separation of Stationary Parametric Gaussian Sources , 2007, IEEE Signal Processing Letters.

[24]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[25]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[26]  Richard Bellman,et al.  Introduction to Matrix Analysis , 1972 .

[27]  A. Belouchrani,et al.  Blind separation of non stationary sources using joint block diagonalization , 2001, Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing (Cat. No.01TH8563).

[28]  Hagit Messer,et al.  Joint Block Diagonalization Algorithms for Optimal Separation of Multidimensional Components , 2012, LVA/ICA.

[29]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[30]  Hao Shen,et al.  Generalised Fastica for Independent Subspace Analysis , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[31]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[32]  L. Lathauwer,et al.  Fetal electrocardiogram extraction by source subspace separation , 1995 .

[33]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[34]  Harold W. Gutch,et al.  Second Order Subspace Analysis and Simple Decompositions , 2010, LVA/ICA.

[35]  Erkki Oja,et al.  Performance analysis of the FastICA algorithm and Crame/spl acute/r-rao bounds for linear independent component analysis , 2006, IEEE Transactions on Signal Processing.

[36]  Dinh-Tuan Pham,et al.  Independent Component Analysis: Separation of non-stationary sources: algorithms and performance , 2001 .

[37]  A. Hyvärinen,et al.  Beyond independent components , 1999 .

[38]  J. Cardoso.  On the Performance of Orthogonal Source Separation Algorithms , 1994 .

[39]  Aapo Hyvärinen,et al.  FastISA: A fast fixed-point algorithm for independent subspace analysis , 2006, ESANN.

[40]  Barnabás Póczos,et al.  ICA and ISA using Schweizer-Wolff measure of dependence , 2008, ICML '08.

[41]  J. Magnus,et al.  The Commutation Matrix: Some Properties and Applications , 1979 .

[42]  Jean-François Cardoso,et al.  Component Separation With Flexible Models—Application to Multichannel Astrophysical Observations , 2008, IEEE Journal of Selected Topics in Signal Processing.

[43]  Jean-Francois Cardoso,et al.  Separation of non stationary sources; achievable performance , 2000, Proceedings of the Tenth IEEE Workshop on Statistical Signal and Array Processing (Cat. No.00TH8496).

[44]  Hagit Messer,et al.  Optimal Performance of Second-Order Multidimensional ICA , 2009, ICA.

[45]  Harold W. Gutch,et al.  Hierarchical Extraction of Independent Subspaces of Unknown Dimensions , 2009, ICA.

[46]  Jean-Francois Cardoso,et al.  THE THREE EASY ROUTES TO INDEPENDENT COMPONENT ANALYSIS; CONTRASTS AND GEOMETRY , 2001 .

[47]  Joos Vandewalle,et al.  Fetal electrocardiogram extraction by blind source subspace separation , 2000, IEEE Transactions on Biomedical Engineering.

[48]  Hagit Messer,et al.  Multidimensional ICA and its performance analysis applied to CMB observations , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[49]  Dinh Tuan Pham,et al.  Joint Approximate Diagonalization of Positive Definite Hermitian Matrices , 2000, SIAM J. Matrix Anal. Appl..

[50]  Barnabás Póczos,et al.  Undercomplete Blind Subspace Deconvolution Via Linear Prediction , 2007, ECML.

[51]  Dorothea Kolossa,et al.  Non-independent BSS: A Model for Evoked MEG Signals with Controllable Dependencies , 2009, ICA.