Theorem for K-Independent Subspace Analysis with Sufficient Conditions

Here, a Separation Theorem about K-Independent Subspace Analysis (K real or complex), a generalization of K-Independent Component Analysis (KICA) is proven. According to the theorem, KISA estimation can be executed in two steps under certain conditions. In the first step, 1-dimensional KICA estimation is executed. In the second step, optimal permutation of the KICA elements is searched for. We present sufficient conditions for the KISA Separation Theorem. Namely, we shall show that (i) spherically symmetric sources (both for real and complex cases), as well as (ii) real 2-dimensional sources invariant to 90 degree rotation, among others, satisfy the conditions of the theorem.

[1]  Erkki Oja,et al.  Independent Component Analysis for Parallel Financial Time Series , 1998, International Conference on Neural Information Processing.

[2]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[3]  Elmar Wolfgang Lang,et al.  Blind source separation and independent component analysis , 2006, Neurocomputing.

[4]  Ah Chung Tsoi,et al.  Blind deconvolution of signals using a complex recurrent network , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[5]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[6]  Fabian J. Theis,et al.  Uniqueness of real and complex linear independent component analysis revisited , 2004, 2004 12th European Signal Processing Conference.

[7]  Visa Koivunen,et al.  Complex random vectors and ICA models: identifiability, uniqueness, and separability , 2005, IEEE Transactions on Information Theory.

[8]  V. Koivunen,et al.  Ieee Workshop on Machine Learning for Signal Processing Complex-valued Ica Using Second , 2022 .

[9]  Fabian J. Theis,et al.  Uniqueness of complex and multidimensional independent component analysis , 2004, Signal Process..

[10]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis of Complex Valued Signals , 2000, Int. J. Neural Syst..

[11]  Simone G. O. Fiori,et al.  Blind separation of circularly distributed sources by neural extended APEX algorithm , 2000, Neurocomputing.

[12]  Shotaro Akaho,et al.  MICA: multimodal independent component analysis , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[13]  Barnabás Póczos,et al.  Cross-Entropy Optimization for Independent Process Analysis , 2006, ICA.

[14]  Scott C. Douglas,et al.  Equivariant Algorithms for Estimating the Strong-Uncorrelating Transform in Complex Independent Component Analysis , 2006, ICA.

[15]  A. Yeredor.  Blind separation of Gaussian sources via second-order statistics with asymptotically optimal weighting , 2000, IEEE Signal Processing Letters.

[16]  Te-Won Lee,et al.  Independent Vector Analysis: An Extension of ICA to Multivariate Components , 2006, ICA.

[17]  M. V. Van Hulle,et al.  Edgeworth Approximation of Multivariate Differential Entropy , 2005, Neural Computation.

[18]  Michael I. Jordan,et al.  FINDING CLUSTERS IN INDEPENDENT COMPONENT ANALYSIS , 2003 .

[19]  Danilo P. Mandic,et al.  Artificial Neural Networks: Formal Models and Their Applications - ICANN 2005 , 2005 .

[20]  Aapo Hyvärinen,et al.  Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation , 1999, Neural Computation.

[21]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[22]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[23]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[24]  Simone G. O. Fiori,et al.  Complex-Weighted One-Unit ‘Rigid-Bodies’ Learning Rule for Independent Component Analysis , 2002, Neural Processing Letters.

[25]  P. Krishnaiah,et al.  Complex elliptically symmetric distributions , 1986 .

[26]  Paris Smaragdis,et al.  Blind separation of convolved mixtures in the frequency domain , 1998, Neurocomputing.

[27]  Jan Eriksson,et al.  Contributions to theory and algorithms of independent component analysis and signal separation , 2004 .

[28]  Fabian J. Theis,et al.  Blind signal separation into groups of dependent signals using joint block diagonalization , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[29]  Barnabás Póczos,et al.  Independent subspace analysis using geodesic spanning trees , 2005, ICML.

[30]  Inder Jeet Taneja,et al.  On Generalized Information Measures and Their Applications , 1989 .

[31]  Barnabás Póczos,et al.  Independent Subspace Analysis Using k-Nearest Neighborhood Distances , 2005, ICANN.

[32]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[33]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[34]  Deniz Erdogmus,et al.  Minimax Mutual Information Approach for ICA of Complex-Valued Linear Mixtures , 2004, ICA.

[35]  Andras Lorincz,et al.  Real and Complex Independent Subspace Analysis by Generalized Variance , 2006 .

[36]  Klaus Obermayer,et al.  Multi Dimensional ICA to Separate Correlated Sources , 2001, NIPS.

[37]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[38]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[39]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[40]  Terrence J. Sejnowski,et al.  Complex Independent Component Analysis of Frequency-Domain Electroencephalographic Data , 2003, Neural Networks.

[41]  Erkki Oja,et al.  Independent Component Analysis for Identification of Artifacts in Magnetoencephalographic Recordings , 1997, NIPS.

[42]  Scott C. Douglas,et al.  Fixed-Point Complex ICA Algorithms for the Blind Separation of Sources Using Their Real or Imaginary Components , 2006, ICA.

[43]  Gabriel Frahm.  Generalized Elliptical Distributions: Theory and Applications , 2004 .

[44]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[45]  S. Kotz,et al.  Symmetric Multivariate and Related Distributions , 1989 .

[46]  Barnabás Póczos,et al.  Independent Subspace Analysis on Innovations , 2005, ECML.

[47]  Vince D. Calhoun,et al.  Complex Infomax: Convergence and Approximation of Infomax with Complex Nonlinearities , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[48]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[49]  Eric Moreau,et al.  An any order generalization of JADE for complex source signals , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).