Local Independent Component Analysis Using Clustering

In standard ICA, a linear data model is used for a global description of the data. Even though linear ICA yields meaningful results in many cases, it can provide a crude approximation only for nonlinear data distributions. We propose a new structure, where local ICA models are used in connection with a suitable clustering algorithm grouping the data. The clustering part is responsible for an overall coarse nonlinear representation of the underlying data, while linear ICA models of each cluster are used for describing local features of the data. The goal is to represent the data better than in linear ICA while avoiding computational diiculties related with nonlinear ICA. We discuss connections to existing methods, and present experimental results for natural image data.

[1]  Colin Fyfe,et al.  Invariant feature maps for analysis of orientations in image data , 1998, ESANN.

[2]  ParraLucas,et al.  Statistical independence and novelty detection with information preserving nonlinear maps , 1996 .

[3]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[4]  C. Gielen,et al.  Neural computation and self-organizing maps, an introduction , 1993 .

[5]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[6]  Kimmo Valkealahti,et al.  Local Independent Component Analysis by the Self-Organizing Map , 1997, ICANN.

[7]  Juan K. Lin.  Factorizing Multivariate Function Classes , 1997, NIPS.

[8]  Erkki Oja,et al.  Neural Independent Component Analysis - Approaches and Applications , 1998 .

[9]  Juan K. Lin,et al.  Faithful Representation of Separable Distributions , 1997, Neural Computation.

[10]  J. Príncipe,et al.  Entropy manipulation of arbitrary nonlinear mappings , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[11]  Juha Karhunen,et al.  Neural approaches to independent component analysis and source separation , 1996, ESANN.

[12]  Aapo Hyvärinen,et al.  ON EXISTENCE AND UNIQUENESS OF SOLUTIONS IN NON-LINEAR INDEPENDENT COMPONENT ANALYSIS , 1998 .

[13]  Lei Xu,et al.  Bayesian Ying-Yang machine, clustering and number of clusters , 1997, Pattern Recognit. Lett..

[14]  A. Dale Magoun,et al.  Decision, estimation and classification , 1989 .

[15]  Andrzej Cichocki,et al.  Information-theoretic approach to blind separation of sources in non-linear mixture , 1998, Signal Process..

[16]  Christian Jutten,et al.  Nonlinear source separation: the post-nonlinear mixtures , 1997, ESANN.

[17]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..