Regularised nonlinear blind signal separation using sparsely connected network

A nonlinear approach based on the Tikhonov regularised cost function is presented for blind signal separation of nonlinear mixtures. The proposed approach uses a multilayer perceptron as the nonlinear demixer and combines both information theoretic learning and structural complexity learning into a single framework. It is shown that this approach can be jointly used to extract independent components while constraining the overall perceptron network to be as sparse as possible. The update algorithm for the nonlinear demixer is subsequently derived using the new cost function. Sparseness in the network connection is utilised to determine the total number of layers required in the multilayer perceptron and to prevent the nonlinear demixer from outputting arbitrary independent components. Experiments are meticulously conducted to study the performance of the new approach and the outcomes of these studies are critically assessed for performance comparison with existing methods.

[1]  John E. Moody,et al.  Smoothing Regularizers for Projective Basis Function Networks , 1996, NIPS.

[2]  Aapo Hyvärinen,et al.  Nonlinear independent component analysis: Existence and uniqueness results , 1999, Neural Networks.

[3]  Christian Jutten,et al.  Source separation in post-nonlinear mixtures , 1999, IEEE Trans. Signal Process..

[4]  Andrzej Cichocki,et al.  Information-theoretic approach to blind separation of sources in non-linear mixture , 1998, Signal Process..

[5]  Gilles Burel,et al.  Blind separation of sources: A nonlinear neural algorithm , 1992, Neural Networks.

[6]  Jack D. Cowan,et al.  Source Separation and Density Estimation by Faithful Equivariant SOM , 1996, NIPS.

[7]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[8]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[9]  Jacek M. Zurada,et al.  Nonlinear Blind Source Separation Using a Radial Basis Function Network , 2001 .

[10]  E. Oja,et al.  Nonlinear Blind Source Separation by Variational Bayesian Learning , 2003, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[11]  Te-Won Lee,et al.  Blind source separation of nonlinear mixing models , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[12]  Terrence J. Sejnowski,et al.  Nonlinear blind source separation using kernel feature spaces , 2001 .

[13]  W. L. Woo,et al.  Blind restoration of nonlinearly mixed signals using multilayer polynomial neural network , 2004 .

[14]  M. Girolami,et al.  Advances in Independent Component Analysis , 2000, Perspectives in Neural Computing.

[15]  Raffaele Parisi,et al.  BLIND SOURCE SEPARATION IN NONLINEAR MIXTURES BY ADAPTIVE SPLINE NEURAL NETWORKS , 2001 .

[16]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[17]  J. Karhunen,et al.  Advances in Nonlinear Blind Source Separation , 2003 .

[18]  Eduardo D. Sontag,et al.  Feedback Stabilization Using Two-Hidden-Layer Nets , 1991, 1991 American Control Conference.

[19]  E. Oja,et al.  Independent Component Analysis , 2013 .

[20]  Dominique Martinez,et al.  Nonlinear blind source separation using kernels , 2003, IEEE Trans. Neural Networks.

[21]  Anisse Taleb,et al.  A generic framework for blind source separation in structured nonlinear models , 2002, IEEE Trans. Signal Process..