Effective Blind Source Separation Based on the Adam Algorithm

In this paper, we derive a modified InfoMax algorithm for the solution of Blind Signal Separation (BSS) problems by using advanced stochastic methods. The proposed approach is based on a novel stochastic optimization approach known as the Adaptive Moment Estimation (Adam) algorithm. The proposed BSS solution can benefit from the excellent properties of the Adam approach. In order to derive the new learning rule, the Adam algorithm is introduced in the derivation of the cost function maximization in the standard InfoMax algorithm. The natural gradient adaptation is also considered. Finally, some experimental results show the effectiveness of the proposed approach.

[1]  Michele Scarpiniti,et al.  Generalized splitting functions for blind separation of complex signals , 2008, Neurocomputing.

[2]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[3]  Paris Smaragdis,et al.  Blind separation of convolved mixtures in the frequency domain , 1998, Neurocomputing.

[4]  Shoko Araki,et al.  The fundamental limitation of frequency domain blind source separation for convolutive mixtures of speech , 2003, IEEE Trans. Speech Audio Process..

[5]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[6]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[7]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[8]  Francesco Carlo Morabito,et al.  Wavelet-ICA methodology for efficient artifact removal from Electroencephalographic recordings , 2007, 2007 International Joint Conference on Neural Networks.

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Scott C. Douglas,et al.  Scaled Natural Gradient Algorithms for Instantaneous and Convolutive Blind Source Separation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[11]  Da-Zheng Feng,et al.  Adaptive Improved Natural Gradient Algorithm for Blind Source Separation , 2009, Neural Computation.

[12]  Soo-Young Lee.  Blind Source Separation and Independent Component Analysis: A Review , 2005 .

[13]  Seungjin Choi.  Blind Source Separation and Independent Component Analysis : A Review , 2004 .

[14]  E. Moreau,et al.  On convolutive Blind Source Separation in a noisy context and a total variation regularization , 2010, 2010 IEEE 11th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[15]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[16]  Thomas P. von Hoff,et al.  STEP-SIZE CONTROL IN BLIND SOURCE SEPARATION , 2000 .

[17]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[18]  Razvan Pascanu,et al.  Revisiting Natural Gradient for Deep Networks , 2013, ICLR.

[19]  Michele Scarpiniti,et al.  Flexible Nonlinear Blind Signal Separation in the Complex Domain , 2008, Int. J. Neural Syst..