Independent Slow Feature Analysis and Nonlinear Blind Source Separation

In the linear case, statistical independence is a sufficient criterion for performing blind source separation. In the nonlinear case, however, it leaves an ambiguity in the solutions that has to be resolved by additional criteria. Here we argue that temporal slowness complements statistical independence well and that a combination of the two leads to unique solutions of the nonlinear blind source separation problem. The algorithm we present is a combination of second-order independent component analysis and slow feature analysis and is referred to as independent slow feature analysis. Its performance is demonstrated on nonlinearly mixed music data. We conclude that slowness is indeed a useful complement to statistical independence but that time-delayed second-order moments are only a weak measure of statistical independence.

[1]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[2]  Laurenz Wiskott,et al.  Understanding Slow Feature Analysis: A Mathematical Framework , 2008 .

[3]  Christian Jutten,et al.  Nonlinear source separation: the post-nonlinear mixtures , 1997, ESANN.

[4]  Andreas Ziehe,et al.  Blind separation of post-nonlinear mixtures using gaussianizing transformations and temporal decorrelation , 2003 .

[5]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[6]  Motoaki Kawanabe,et al.  Kernel-Based Nonlinear Blind Source Separation , 2003, Neural Computation.

[7]  Schuster,et al.  Separation of a mixture of independent signals using time delayed correlations. , 1994, Physical review letters.

[8]  Christian Jutten,et al.  Source separation in post-nonlinear mixtures , 1999, IEEE Trans. Signal Process..

[9]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[10]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[11]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[12]  Aapo Hyvärinen,et al.  Nonlinear independent component analysis: Existence and uniqueness results , 1999, Neural Networks.

[13]  Jean-Francois Cardoso,et al.  THE THREE EASY ROUTES TO INDEPENDENT COMPONENT ANALYSIS; CONTRASTS AND GEOMETRY , 2001 .

[14]  Andrzej Cichocki,et al.  Information-theoretic approach to blind separation of sources in non-linear mixture , 1998, Signal Process..

[15]  Andreas Ziehe,et al.  TDSEP { an e(cid:14)cient algorithm for blind separation using time structure , 1998 .

[16]  T. Rao,et al.  Tensor Methods in Statistics , 1989 .

[17]  Anisse Taleb,et al.  A generic framework for blind source separation in structured nonlinear models , 2002, IEEE Trans. Signal Process..

[18]  J. Karhunen,et al.  Advances in Nonlinear Blind Source Separation , 2003 .

[19]  Motoaki Kawanabe,et al.  Blind Separation of Post-nonlinear Mixtures using Linearizing Transformations and Temporal Decorrelation , 2003, J. Mach. Learn. Res..

[20]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources , 1999, Neural Comput..

[21]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[22]  Luís B. Almeida,et al.  Linear and nonlinear ICA based on mutual information - the MISEP method , 2004, Signal Process..

[23]  Christian Jutten,et al.  A geometric approach for separating post non-linear mixtures , 2002, 2002 11th European Signal Processing Conference.

[24]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[25]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[26]  Laurenz Wiskott,et al.  CuBICA: independent component analysis by simultaneous third- and fourth-order cumulant diagonalization , 2004, IEEE Transactions on Signal Processing.

[27]  Shun-ichi Amari,et al.  Recurrent Neural Networks For Blind Separation of Sources , 1995 .

[28]  Lang Tong,et al.  Indeterminacy and identifiability of blind identification , 1991 .

[29]  Laurenz Wiskott,et al.  What Is the Relation Between Slow Feature Analysis and Independent Component Analysis? , 2006, Neural Computation.

[30]  C. Jutten,et al.  On the separability of nonlinear mixtures of temporally correlated sources , 2003, IEEE Signal Processing Letters.

[31]  Yujie Zhang,et al.  Linear and nonlinear ICA based on mutual information , 2007, 2007 International Symposium on Intelligent Signal Processing and Communication Systems.