Some global and local convergence analysis on the information-theoretic independent component analysis approach

Abstract In this paper, we present a detailed theoretical analysis on the information-theoretic Independent Component Analysis (IT-ICA) approach. We first provide a number of lemmas and theorems on properties of the corresponding cost function in the general n-channel case with differentiable, odd, monotonic decreasing nonlinearity. A theorem on behaviour of the cost function along a radially outward line is given for characterizing the global configuration of the cost function in the parameter space. Furthermore, on the 2-channel IT-ICA system with cubic nonlinearity, we not only exhaustively solve out all equilibrium points and the condition for stability, but also give a global convergence theorem.

[1]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[2]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[3]  Dinh Tuan Pham,et al.  Separation of a mixture of independent sources through a maximum likelihood approach , 1992 .

[4]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[5]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[6]  Pierre Comon,et al.  Blind separation of sources, part II: Problems statement , 1991, Signal Process..

[7]  Esfandiar Sorouchyari,et al.  Blind separation of sources, part III: Stability analysis , 1991, Signal Process..

[8]  Eric Moreau,et al.  New self-adaptative algorithms for source separation based on contrast functions , 1993, [1993 Proceedings] IEEE Signal Processing Workshop on Higher-Order Statistics.

[9]  Yves Deville,et al.  Application of blind source separation techniques to multi-tag contactless identification systems , 1995 .

[10]  Tianping Chen,et al.  A neural network approach to blind identification of stochastic and deterministic signals , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[11]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[12]  Shun-ichi Amari,et al.  Independent component analysis by the information-theoretic approach with mixture of densities , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[13]  Shun-ichi Amari,et al.  Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information , 1997, Neural Computation.

[14]  Eric Moreau,et al.  Self-adaptive source separation .I. Convergence analysis of a direct linear network controlled by the Herault-Jutten algorithm , 1997, IEEE Trans. Signal Process..

[15]  Andrzej Cichocki,et al.  New learning algorithm for blind separation of sources , 1992 .

[16]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[17]  Shun-ichi Amari,et al.  Learned parametric mixture based ICA algorithm , 1998, Neurocomputing.

[18]  J. Karhunen,et al.  A bigradient optimization approach for robust PCA, MCA, and source separation , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[19]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[20]  Eric Moreau,et al.  Self-adaptive source separation. II. Comparison of the direct, feedback, and mixed linear network , 1998, IEEE Trans. Signal Process..

[21]  Juha Karhunen,et al.  Neural approaches to independent component analysis and source separation , 1996, ESANN.