Deterministic Independent Component Analysis

We study independent component analysis with noisy observations. We present, for the first time in the literature, consistent, polynomial-time algorithms to recover non-Gaussian source signals and the mixing matrix with a reconstruction error that vanishes at a 1/√T rate using T observations and scales only polynomially with the natural parameters of the problem. Our algorithms and analysis also extend to deterministic source signals whose empirical distributions are approximately independent.

[1]  J. Hüsler.  Minimal spacings of non-uniform densities , 1987 .

[2]  G. W. Stewart,et al.  Computer Science and Scientific Computing , 1990 .

[3]  V. N. Bogaevski,et al.  Matrix Perturbation Theory , 1991 .

[4]  Alan M. Frieze,et al.  Learning linear transformations , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[5]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[6]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[7]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[8]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[9]  Visa Koivunen,et al.  Characteristic-function-based independent component analysis , 2003, Signal Process..

[10]  A. Tsybakov,et al.  Nonparametric independent component analysis , 2004 .

[11]  P. Bickel,et al.  Consistent independent component analysis and prewhitening , 2005, IEEE Transactions on Signal Processing.

[12]  Lorenzo Rosasco,et al.  Learning from Examples as an Inverse Problem , 2005, J. Mach. Learn. Res..

[13]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[14]  Erkki Oja,et al.  Performance analysis of the FastICA algorithm and Crame/spl acute/r-rao bounds for linear independent component analysis , 2006, IEEE Transactions on Signal Processing.

[15]  P. Bickel,et al.  Efficient independent component analysis , 2006, 0705.4230.

[16]  Erkki Oja,et al.  The FastICA Algorithm Revisited: Convergence Analysis , 2006, IEEE Transactions on Neural Networks.

[17]  E. Oja,et al.  Corrections to "Performance Analysis of the FastICA Algorithm and Cram´ er-Rao Bounds for Linear Independent Component Analysis", TSP 04/06 , 2007 .

[18]  Esa Ollila,et al.  The Deflation-Based FastICA Estimator: Statistical Analysis Revisited , 2010, IEEE Transactions on Signal Processing.

[19]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[20]  E. Oja,et al.  Performance Analysis of the FastICA Algorithm and Cramér – Rao Bounds for Linear Independent Component Analysis , 2010 .

[21]  A. Dasgupta.  Finite Sample Theory of Order Statistics and Extremes , 2011 .

[22]  Barnabás Póczos,et al.  Separation theorem for independent subspace analysis and its consequences , 2012, Pattern Recognit..

[23]  Sham M. Kakade,et al.  Learning Gaussian Mixture Models: Moment Methods and Spectral Decompositions , 2012, arXiv.org.

[24]  Anima Anandkumar,et al.  A Method of Moments for Mixture Models and Hidden Markov Models , 2012, COLT.

[25]  Csaba Szepesvári,et al.  Statistical linear estimation with penalized estimators: an application to reinforcement learning , 2012, ICML.

[26]  Tianwen Wei,et al.  FastICA Algorithm: Five Criteria for the Optimal Choice of the Nonlinearity Function , 2013, IEEE Transactions on Signal Processing.

[27]  Sham M. Kakade,et al.  Learning mixtures of spherical gaussians: moment methods and spectral decompositions , 2012, ITCS '13.

[28]  Anima Anandkumar,et al.  Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updates , 2014, ArXiv.

[29]  Santosh S. Vempala,et al.  Max vs Min: Independent Component Analysis with nearly Linear Sample Complexity , 2014, ArXiv.

[30]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[31]  Tianwen Wei.  A study of the fixed points and spurious solutions of the FastICA algorithm , 2014, 1408.6693.

[32]  Santosh S. Vempala,et al.  Fourier PCA and robust tensor decomposition , 2013, STOC.

[33]  Tianwen Wei,et al.  A Convergence and Asymptotic Analysis of the Generalized Symmetric FastICA Algorithm , 2014, IEEE Transactions on Signal Processing.

[34]  Tianwen Wei,et al.  A Convergence and Asymptotic Analysis of the Generalized Symmetric FastICA Algorithm , 2014, IEEE Transactions on Signal Processing.

[35]  Sanjeev Arora,et al.  Provable ICA with Unknown Gaussian Noise, and Implications for Gaussian Mixtures and Autoencoders , 2012, Algorithmica.

[36]  K. Nordhausen,et al.  Fourth Moments and Independent Component Analysis , 2014, 1406.4765.