Independent component analysis based on marginal density estimation using weighted Parzen windows

This work proposes a novel algorithm for independent component analysis (ICA) based on marginal density estimation. The proposed ICA algorithm aims to search for an effective demixing matrix as well as weighted Parzen window (WPW) representations for marginal densities of independent components so as to express a factorial joint density for high dimensional observations. Following the linear mixture assumption, independent component analysis is mathematically translated to minimizing the Kullback-Leibler (KL) divergence of independent components. By using Potts encoding, we express the KL divergence in an approximating form, which is shown to be tractable with respect to the WPW parameters as well as the demixing matrix and can be minimized by two interactive dynamic modules derived by the annealed expectation-maximization method and the natural gradient descent method, respectively. By numerical simulations, we test the proposed ICA algorithm with observations separately sampled from linear mixtures of independent sources and real world signals, including fetal electrocardiograms, mixed facial images and event-related potentials, extensively showing its accuracy and reliability for independent component analysis in comparison with some other popular ICA algorithms.

[1]  Fu-Liang Yin,et al.  Blind source separation applied to image cryptosystems with dual encryption , 2002 .

[2]  Cheng-Yuan Liou,et al.  Self-organization using Potts models , 1996, Neural Networks.

[3]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[4]  Shun-ichi Amari,et al.  Blind Separation of a Mixture of Uniformly Distributed Source Signals: A Novel Approach , 1999, Neural Computation.

[5]  Asoke K. Nandi,et al.  Noninvasive fetal electrocardiogram extraction: blind separation versus adaptive noise cancellation , 2001, IEEE Transactions on Biomedical Engineering.

[6]  Masud Mansuripur,et al.  Introduction to information theory , 1986 .

[7]  Chao He,et al.  Probability Density Estimation from Optimally Condensed Data Samples , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Magnus Rattray,et al.  Natural gradient descent for on-line learning , 1998 .

[9]  G. McCarthy,et al.  Human Brain Mapping 6:1–13(1998) � Functional MRI Studies of Auditory Comprehension , 2022 .

[10]  Ben J. A. Kröse,et al.  Efficient Greedy Learning of Gaussian Mixture Models , 2003, Neural Computation.

[11]  Aapo Hyvärinen,et al.  Validating the independent components of neuroimaging time series via clustering and visualization , 2004, NeuroImage.

[12]  Jiann-Ming Wu,et al.  Learning generative models of natural images , 2002, Neural Networks.

[13]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[14]  Terrence J. Sejnowski,et al.  Blind separation and blind deconvolution: an information-theoretic approach , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[15]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[16]  Joos Vandewalle,et al.  Fetal electrocardiogram extraction by blind source subspace separation , 2000, IEEE Transactions on Biomedical Engineering.

[17]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[18]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[19]  S Makeig,et al.  Blind separation of auditory event-related brain responses into independent components. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[20]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[21]  Jean-Francois Cardoso,et al.  Blind signal separation: statistical principles , 1998, Proc. IEEE.

[22]  Vwani P. Roychowdhury,et al.  Independent component analysis based on nonparametric density estimation , 2004, IEEE Transactions on Neural Networks.

[23]  Octavia I. Camps,et al.  Weighted Parzen Windows for Pattern Classification , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Jiann-Ming Wu,et al.  Independent component analysis using Potts models , 2001, IEEE Trans. Neural Networks.

[25]  S Makeig,et al.  Analysis of fMRI data by blind separation into independent spatial components , 1998, Human brain mapping.

[26]  Carsten Peterson,et al.  A New Method for Mapping Optimization Problems Onto Neural Networks , 1989, Int. J. Neural Syst..