Independent Component Analysis Using Multilayer Networks

A basic element in most independent component analysis (ICA) algorithms is the choice of a model for the score functions of the unknown sources. In this letter, a novel ICA algorithm is proposed, which is truly blind to the particular underlying distribution of the mixed signals. Using a multilayer network density estimation technique, the algorithm reconstructs score functions of the source signals. We show with experiments involving linear mixtures of various source signals with different statistical characteristics that the new algorithm not only outperforms state-of-the-art ICA methods but also our approach only requires a fraction of the sample sizes in order to outperform methods with partially adaptive score functions.

[1]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[2]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources , 1999, Neural Comput..

[3]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[4]  J. Nadal Non linear neurons in the low noise limit : a factorial code maximizes information transferJean , 1994 .

[5]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[6]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[7]  Amir F. Atiya,et al.  Density estimation and random variate generation using multilayer networks , 2002, IEEE Trans. Neural Networks.

[8]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[9]  Vwani P. Roychowdhury,et al.  Independent component analysis based on nonparametric density estimation , 2004, IEEE Transactions on Neural Networks.

[10]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[11]  Visa Koivunen,et al.  Pearson System based Method for Blind Separation , 2000 .

[12]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[13]  Nikos A. Vlassis,et al.  Efficient source adaptivity in independent component analysis , 2001, IEEE Trans. Neural Networks.

[14]  Visa Koivunen,et al.  Blind separation methods based on Pearson system and its extensions , 2002, Signal Process..

[15]  Andrzej Cichocki,et al.  Stability Analysis of Learning Algorithms for Blind Source Separation , 1997, Neural Networks.

[16]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[17]  Néstor Parga,et al.  Information processing by a perceptron in an unsupervised learning task , 1993 .

[18]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .