Overlearning in Marginal Distribution-Based ICA: Analysis and Solutions

The present paper is written as a word of caution, with users of independent component analysis (ICA) in mind, to overlearning phenomena that are often observed. We consider two types of overlearning, typical to high-order statistics based ICA. These algorithms can be seen to maximise the negentropy of the source estimates. The first kind of overlearning results in the generation of spike-like signals, if there are not enough samples in the data or there is a considerable amount of noise present. It is argued that, if the data has power spectrum characterised by 1= f curve, we face a more severe problem, which cannot be solved inside the strict ICA model. This overlearning is better characterised by bumps instead of spikes. Both overlearning types are demonstrated in the case of artificial signals as well as magnetoencephalograms (MEG). Several methods are suggested to circumvent both types, either by making the estimation of the ICA model more robust or by including further modelling of the data.

[1]  Matthew J. Beal,et al.  Graphical Models and Variational Methods , 2001 .

[2]  Terrence J. Sejnowski,et al.  Variational Learning of Clusters of Undercomplete Nonsymmetric Independent Components , 2003, J. Mach. Learn. Res..

[3]  Shun-ichi Amari,et al.  Estimating Functions of Independent Component Analysis for Temporally Correlated Signals , 2000, Neural Computation.

[4]  Aapo Hyvärinen,et al.  Complexity Pursuit: Separating Interesting Components from Time Series , 2001, Neural Computation.

[5]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[6]  Barak A. Pearlmutter,et al.  Independent Components of Magnetoencephalography: Localization , 2002, Neural Computation.

[7]  Motoaki Kawanabe,et al.  A resampling approach to estimate the stability of one-dimensional or multidimensional independent components , 2002, IEEE Transactions on Biomedical Engineering.

[8]  Andreas Ziehe,et al.  TDSEP { an e(cid:14)cient algorithm for blind separation using time structure , 1998 .

[9]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[10]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[11]  A. J. Bell,et al.  INDEPENDENT COMPONENT ANALYSIS OF BIOMEDICAL SIGNALS , 2000 .

[12]  Erkki Oja,et al.  Image Feature Extraction Using Independent Component Analysis , 1996 .

[13]  Erkki Oja,et al.  Independent component approach to the analysis of EEG and MEG recordings , 2000, IEEE Transactions on Biomedical Engineering.

[14]  P O Hoyer,et al.  Independent component analysis applied to feature extraction from colour and stereo images , 2000, Network.

[15]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[16]  J. W. Miskin,et al.  Ensemble Learning for Blind Source Separation , 2001 .

[17]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[18]  R. Ilmoniemi,et al.  Magnetoencephalography-theory, instrumentation, and applications to noninvasive studies of the working human brain , 1993 .

[19]  Andreas Ziehe,et al.  Unmixing Hyperspectral Data , 1999, NIPS.

[20]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[21]  Harri Valpola,et al.  Accurate, Fast and Stable Denoising Source Separation Algorithms , 2004, ICA.

[22]  H. Attias.  Independent Component Analysis: ICA, graphical models and variational methods , 2001 .

[23]  R. A. Choudrey.  FLEXIBLE BAYESIAN INDEPENDENT COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION , 2001 .

[24]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[25]  Erkki Oja,et al.  ARTEFACT DETECTION IN ASTROPHYSICAL IMAGE DATA USING INDEPENDENT COMPONENT ANALYSIS , 2001 .

[26]  Erkki Oja,et al.  Independent Component Analysis for Identification of Artifacts in Magnetoencephalographic Recordings , 1997, NIPS.

[27]  E. Oja,et al.  Independent Component Analysis , 2013 .

[28]  Jean-Francois Cardoso,et al.  Source separation using higher order moments , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[29]  P. Bak,et al.  Self-organized criticality. , 1988, Physical review. A, General physics.

[30]  D. Sornette,et al.  Data-adaptive wavelets and multi-scale singular-spectrum analysis , 1998, chao-dyn/9810034.

[31]  Aapo Hyvärinen,et al.  Topographic Independent Component Analysis , 2001, Neural Computation.