Independent Subspace Analysis can Cope with the 'Curse of Dimensionality'

We search for hidden independent components, in particular we consider the independent subspace analysis (ISA) task. Earlier ISA procedures assume that the dimensions of the components are known. Here we show a method that enables the non-combinatorial estimation of the components. We make use of a decomposition principle called the ISA separation theorem. According to this separation theorem the ISA task can be reduced to the independent component analysis (ICA) task that assumes one-dimensional components and then to a grouping procedure that collects the respective nonindependent elements into independent groups. We show that non-combinatorial grouping is feasible by means of the non-linear f-correlation matrices between the estimated components.

[1]  Erkki Oja,et al.  Independent Component Analysis for Parallel Financial Time Series , 1998, International Conference on Neural Information Processing.

[2]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[3]  Shotaro Akaho,et al.  MICA: multimodal independent component analysis , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[4]  Barnabás Póczos,et al.  Cross-Entropy Optimization for Independent Process Analysis , 2006, ICA.

[5]  Fabian J. Theis,et al.  Blind signal separation into groups of dependent signals using joint block diagonalization , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[6]  Barnabás Póczos,et al.  Undercomplete Blind Subspace Deconvolution , 2007, J. Mach. Learn. Res..

[7]  Barnabás Póczos,et al.  Non-combinatorial estimation of independent autoregressive sources , 2006, Neurocomputing.

[8]  Erkki Oja,et al.  Independent Component Analysis for Identification of Artifacts in Magnetoencephalographic Recordings , 1997, NIPS.

[9]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[10]  Fabian J. Theis,et al.  Uniqueness of complex and multidimensional independent component analysis , 2004, Signal Process..

[11]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[12]  Michael I. Jordan,et al.  Beyond Independent Components: Trees and Clusters , 2003, J. Mach. Learn. Res..

[13]  Fabian J. Theis,et al.  Towards a general independent subspace analysis , 2006, NIPS.

[14]  Danilo P. Mandic,et al.  Artificial Neural Networks: Formal Models and Their Applications - ICANN 2005 , 2005 .

[15]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[16]  Andreas Ziehe,et al.  Identifying interactions in mixed and noisy complex systems. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[18]  Barnabás Póczos,et al.  Independent subspace analysis using geodesic spanning trees , 2005, ICML.

[19]  Barnabás Póczos,et al.  Independent Subspace Analysis Using k-Nearest Neighborhood Distances , 2005, ICANN.

[20]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[21]  Aapo Hyvärinen,et al.  Topographic Independent Component Analysis , 2001, Neural Computation.

[22]  Aapo Hyvärinen,et al.  Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation , 1999, Neural Computation.

[23]  Aapo Hyvärinen,et al.  FastISA: A fast fixed-point algorithm for independent subspace analysis , 2006, ESANN.

[24]  Klaus Obermayer,et al.  Multi Dimensional ICA to Separate Correlated Sources , 2001, NIPS.

[25]  Barnabás Póczos,et al.  Independent Subspace Analysis on Innovations , 2005, ECML.

[26]  Barnabas Poczos,et al.  Theorem for K-Independent Subspace Analysis with Sufficient Conditions , 2006, math/0608100.

[27]  Andras Lorincz,et al.  Real and Complex Independent Subspace Analysis by Generalized Variance , 2006 .

[28]  Alexander Kraskov,et al.  Least-dependent-component analysis based on mutual information. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.