Adaptive Improved Natural Gradient Algorithm for Blind Source Separation

We propose an adaptive improved natural gradient algorithm for blind separation of independent sources. First, inspired by the well-known backpropagation algorithm, we incorporate a momentum term into the natural gradient learning process to accelerate the convergence rate and improve the stability. Then an estimation function for the adaptation of the separation model is obtained to adaptively control a step-size parameter and a momentum factor. The proposed natural gradient algorithm with variable step-size parameter and variable momentum factor is therefore particularly well suited to blind source separation in a time-varying environment, such as an abruptly changing mixing matrix or signal power. The expected improvement in the convergence speed, stability, and tracking ability of the proposed algorithm is demonstrated by extensive simulation results in both time-invariant and time-varying environments. The ability of the proposed algorithm to separate extremely weak or badly scaled sources is also verified. In addition, simulation results show that the proposed algorithm is suitable for separating mixtures of many sources (e.g., the number of sources is 10) in the complete case.

[1]  V. John Mathews,et al.  A stochastic gradient adaptive filter with gradient adaptive step size , 1993, IEEE Trans. Signal Process..

[2]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[3]  Jianye Sun,et al.  Local coupled feedforward neural network , 2010, Neural Networks.

[4]  Scott C. Douglas,et al.  Scaled Natural Gradient Algorithms for Instantaneous and Convolutive Blind Source Separation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[5]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[6]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[7]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[8]  Hiroshi Sawada,et al.  Natural gradient multichannel blind deconvolution and speech separation using causal FIR filters , 2004, IEEE Transactions on Speech and Audio Processing.

[9]  Shun-ichi Amari,et al.  Improving Generalization Performance of Natural Gradient Learning Using Optimized Regularization by NIC , 2004, Neural Computation.

[10]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[11]  Michael S. Brandstein,et al.  Microphone Arrays - Signal Processing Techniques and Applications , 2001, Microphone Arrays.

[12]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[13]  Steve McLaughlin,et al.  Variable step-size EASI algorithm for sequential blind source separation , 2004 .

[14]  Tom E. Bishop,et al.  Blind Image Restoration Using a Block-Stationary Signal Model , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[15]  Te-Won Lee,et al.  Independent Component Analysis , 1998, Springer US.

[16]  Thomas P. von Hoff,et al.  STEP-SIZE CONTROL IN BLIND SOURCE SEPARATION , 2000 .

[17]  Geoffrey E. Hinton,et al.  Learning representations by back-propagation errors, nature , 1986 .

[18]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .

[19]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[20]  Francesco Piazza,et al.  Stability Analysis of Natural Gradient Learning Rules in Complete ICA: A Unifying Perspective , 2007, IEEE Signal Processing Letters.

[21]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[22]  Magnus Rattray,et al.  Natural gradient descent for on-line learning , 1998 .

[23]  Shun-ichi Amari,et al.  Stability Analysis Of Adaptive Blind Source Separation , 1997 .

[24]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[25]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[26]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[27]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[28]  Andrzej Cichocki,et al.  Stability Analysis of Learning Algorithms for Blind Source Separation , 1997, Neural Networks.

[29]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[30]  Kenji Fukumizu,et al.  Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons , 2000, Neural Computation.

[31]  Wenwu Wang,et al.  Variable step-size sign natural gradient algorithm for sequential blind source separation , 2005, IEEE Signal Processing Letters.

[32]  S. Haykin.  Unsupervised adaptive filtering, vol. 1: Blind source separation , 2000 .

[33]  Andrzej Cichocki,et al.  Nonholonomic Orthogonal Learning Algorithms for Blind Source Separation , 2000, Neural Computation.

[34]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[35]  S.C. Douglas,et al.  Adaptive step size techniques for decorrelation and blind source separation , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[36]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[37]  Paris Smaragdis,et al.  Blind separation of convolved mixtures in the frequency domain , 1998, Neurocomputing.

[38]  Andrzej Cichocki,et al.  Robust learning algorithm for blind separation of signals , 1994 .

[39]  Aapo Hyvärinen,et al.  Nonlinear independent component analysis: Existence and uniqueness results , 1999, Neural Networks.

[40]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .