Least-dependent-component analysis based on mutual information.

We propose to use precise estimators of mutual information (MI) to find the least dependent components in a linearly mixed signal. On the one hand, this seems to lead to better blind source separation than with any other presently available algorithm. On the other hand, it has the advantage, compared to other implementations of "independent" component analysis (ICA), some of which are based on crude approximations for MI, that the numerical values of the MI can be used for (i) estimating residual dependencies between the output components; (ii) estimating the reliability of the output by comparing the pairwise MIs with those of remixed components; and (iii) clustering the output according to the residual interdependencies. For the MI estimator, we use a recently proposed k -nearest-neighbor-based algorithm. For time sequences, we combine this with delay embedding, in order to take into account nontrivial time correlations. After several tests with artificial data, we apply the resulting MILCA (mutual-information-based least dependent component analysis) algorithm to a real-world dataset, the ECG of a pregnant woman.

[1]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[2]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[3]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[4]  R. L. Somorjai,et al.  Methods for Estimating the Intrinsic Dimsnionality of High-Dimensional Point Sets , 1986 .

[5]  Michael I. Jordan,et al.  Beyond Independent Components: Trees and Clusters , 2003, J. Mach. Learn. Res..

[6]  Aapo Hyvärinen,et al.  Icasso: software for investigating the reliability of ICA estimates by clustering and visualization , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[7]  J. Victor Binless strategies for estimation of information from neural data. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  M. Neugebauer,et al.  Reviews of Geophysics , 1988 .

[9]  Motoaki Kawanabe,et al.  A resampling approach to estimate the stability of one-dimensional or multidimensional independent components , 2002, IEEE Transactions on Biomedical Engineering.

[10]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[11]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[12]  John W. Fisher,et al.  ICA Using Spacings Estimates of Entropy , 2003, J. Mach. Learn. Res..

[13]  R. Vautard,et al.  Singular spectrum analysis in nonlinear dynamics, with applications to paleoclimatic time series , 1989 .

[14]  Noboru Murata,et al.  Analysing ICA component by injection noise , 2003 .

[15]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[16]  Michael Ghil,et al.  ADVANCED SPECTRAL METHODS FOR CLIMATIC TIME SERIES , 2002 .

[17]  Wee Chew,et al.  Band-target entropy minimization. A robust algorithm for pure component spectral recovery. Application to complex randomized mixtures of six components. , 2003, Analytical chemistry.

[18]  T. Schreiber,et al.  Information transfer in continuous processes , 2002 .

[19]  W. Windig,et al.  Interactive self-modeling mixture analysis , 1991 .

[20]  Erkki Oja,et al.  A "nonnegative PCA" algorithm for independent component analysis , 2004, IEEE Transactions on Neural Networks.

[21]  J. A. Stewart,et al.  Nonlinear Time Series Analysis , 2015 .

[22]  X. Z. Wang,et al.  A New Approach to Near-Infrared Spectral Data Analysis Using Independent Component Analysis , 2001, J. Chem. Inf. Comput. Sci..

[23]  G. P. King,et al.  Extracting qualitative dynamics from experimental data , 1986 .

[24]  D. T. Kaplan,et al.  Fetal ECG extraction with nonlinear state-space projections , 1998, IEEE Transactions on Biomedical Engineering.

[25]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[26]  Paul Geladi,et al.  Chemometrics in spectroscopy. Part 1. Classical chemometrics , 2003 .

[27]  Schreiber,et al.  Signal separation by nonlinear projections: The fetal electrocardiogram. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[28]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[29]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[30]  Oldrich A Vasicek,et al.  A Test for Normality Based on Sample Entropy , 1976 .

[31]  Romà Tauler,et al.  Chemometrics applied to unravel multicomponent processes and mixtures: Revisiting latest trends in multivariate resolution , 2003 .

[32]  Yukihiro Ozaki,et al.  SELF-MODELING CURVE RESOLUTION (SMCR): PRINCIPLES, TECHNIQUES, AND APPLICATIONS , 2002 .

[33]  F. Chan,et al.  Free radical EPR spectroscopy analysis using blind source separation. , 2004, Journal of magnetic resonance.

[34]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[35]  H. J. Mclaughlin,et al.  Learn , 2002 .

[36]  Peter Grassberger,et al.  Generalizations of the Hausdorff dimension of fractal measures , 1985 .

[37]  Schuster,et al.  Separation of a mixture of independent signals using time delayed correlations. , 1994, Physical review letters.

[38]  Chris W. Brown,et al.  Self-Modeling Mixture Analysis by Interactive Principal Component Analysis , 2000 .

[39]  P. Philips,et al.  JADETD : COMBINING HIGHER-ORDER STATISTICS AND TEMPORALINFORMATION FOR BLIND SOURCE SEPARATION ( WITH NOISE ) , 1999 .

[40]  Andreas Ziehe,et al.  A fast algorithm for joint diagonalization with application to blind source separation – BLISS Technical Report ∗ – , 2003 .

[41]  Andreas Ziehe,et al.  A Fast Algorithm for Joint Diagonalization with Non-orthogonal Transformations and its Application to Blind Source Separation , 2004, J. Mach. Learn. Res..

[42]  Mw Hirsch,et al.  Chaos In Dynamical Systems , 2016 .

[43]  Andreas Ziehe,et al.  TDSEP { an e(cid:14)cient algorithm for blind separation using time structure , 1998 .

[44]  Te-Won Lee,et al.  An information-theoretic methodology for the resolution of pure component spectra without prior information using spectroscopic measurements , 2004 .

[45]  Bart De Schutter,et al.  DAISY : A database for identification of systems , 1997 .

[46]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[47]  L. Lathauwer,et al.  Fetal electrocardiogram extraction by source subspace separation , 1995 .

[48]  Michael Ghil,et al.  ADAPTIVE FILTERING AND PREDICTION OF NOISY MULTIVARIATE SIGNALS: AN APPLICATION TO SUBANNUAL VARIABILITY IN ATMOSPHERIC ANGULAR MOMENTUM , 1993 .

[49]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixture of sources based on order statistics , 2000, IEEE Trans. Signal Process..

[50]  R. Vautard,et al.  Singular-spectrum analysis: a toolkit for short, noisy chaotic signals , 1992 .

[51]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  Andrew G. Glen,et al.  APPL , 2001 .