Grading learning for blind source separation

By generalizing the learning rate parameter to a learning rate matrix, this paper proposes a grading learning algorithm for blind source separation. The whole learning process is divided into three stages: initial stage, capturing stage and tracking stage. In different stages, different learning rates are used for each output component, which is determined by its dependency on other output components. It is shown that the grading learning algorithm is equivariant and can keep the separating matrix from becoming singular. Simulations show that the proposed algorithm can achieve faster convergence, better steady-state performance and higher numerical robustness, as compared with the existing algorithms using fixed, time-descending and adaptive learning rates.

[1]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[2]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[3]  H. H. Yang,et al.  Serial updating rule for blind separation derived from the method of scoring , 1999, IEEE Trans. Signal Process..

[4]  Andreas Ziehe,et al.  Adaptive On-line Learning in Changing Environments , 1996, NIPS.

[5]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[6]  Andrzej Cichocki,et al.  Flexible Independent Component Analysis , 2000, J. VLSI Signal Process..

[7]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[8]  Shun-ichi Amari,et al.  Self-adaptive neural networks for blind separation of sources , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[9]  S.C. Douglas,et al.  Adaptive step size techniques for decorrelation and blind source separation , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[10]  V. John Mathews,et al.  A stochastic gradient adaptive filter with gradient adaptive step size , 1993, IEEE Trans. Signal Process..

[11]  Sergio Cruces,et al.  An iterative inversion approach to blind source separation , 2000, IEEE Trans. Neural Networks Learn. Syst..

[12]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[13]  Erkki Oja,et al.  The nonlinear PCA criterion in blind source separation: Relations with other approaches , 1998, Neurocomputing.

[14]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[15]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[16]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[17]  Shun-ichi Amari,et al.  Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information , 1997, Neural Computation.

[18]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[19]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixture of sources via an independent component analysis , 1996, IEEE Trans. Signal Process..

[20]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .