From blind signal extraction to blind instantaneous signal separation: criteria, algorithms, and stability

This paper reports a study on the problem of the blind simultaneous extraction of specific groups of independent components from a linear mixture. This paper first presents a general overview and unification of several information theoretic criteria for the extraction of a single independent component. Then, our contribution fills the theoretical gap that exists between extraction and separation by presenting tools that extend these criteria to allow the simultaneous blind extraction of subsets with an arbitrary number of independent components. In addition, we analyze a family of learning algorithms based on Stiefel manifolds and the natural gradient ascent, present the nonlinear optimal activations (score) functions, and provide new or extended local stability conditions. Finally, we illustrate the performance and features of the proposed approach by computer-simulation experiments.

[1]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[2]  Shun-ichi Amari,et al.  Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information , 1997, Neural Computation.

[3]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[4]  Shun-ichi Amari,et al.  Sequential blind signal extraction in order specified by stochastic properties , 1997 .

[5]  Eric Moreau,et al.  High order contrasts for self-adaptive source separation criteria for complex source separation , 1996 .

[6]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[7]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[8]  Simon Haykin,et al.  Entropic Contrasts for Source Separation: Geometry and Stability , 2022 .

[9]  Ali Mansour,et al.  Blind Separation of Sources , 1999 .

[10]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[11]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[12]  Sergio Cruces,et al.  The Minimum Entropy and Cumulants Based Contrast Functions for Blind Source Extraction , 2001, IWANN.

[13]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[14]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[15]  Ruey-Wen Liu,et al.  General approach to blind source separation , 1996, IEEE Trans. Signal Process..

[16]  J. Friedman,et al.  PROJECTION PURSUIT DENSITY ESTIMATION , 1984 .

[17]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[18]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[19]  S. Klinke,et al.  Exploratory Projection Pursuit , 1995 .

[20]  Shun-ichi Amari,et al.  Natural Gradient Learning for Over- and Under-Complete Bases in ICA , 1999, Neural Computation.

[21]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[22]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[23]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[24]  J. Nadal.  Non linear neurons in the low noise limit : a factorial code maximizes information transferJean , 1994 .

[25]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[26]  M. Girolami.  Negentropy and Kurtosis as Projection Pursuit Indices Provide Generalised ICA Algorithms , 1996, NIPS 1996.

[27]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[28]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  Shun-ichi Amari,et al.  Adaptive blind signal processing-neural network approaches , 1998, Proc. IEEE.

[31]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[32]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[33]  Sergio Cruces,et al.  On a new blind signal extraction algorithm: different criteria and stability analysis , 2002, IEEE Signal Processing Letters.

[34]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[35]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..