Fast and robust fixed-point algorithms for independent component analysis

Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. In this paper, we use a combination of two different approaches for linear ICA: Comon's information-theoretic approach and the projection pursuit approach. Using maximum entropy approximations of differential entropy, we introduce a family of new contrast (objective) functions for ICA. These contrast functions enable both the estimation of the whole decomposition by minimizing mutual information, and estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model, and it is shown how to choose contrast functions that are robust and/or of minimum variance. Finally, we introduce simple fixed-point algorithms for practical optimization of the contrast functions. These algorithms optimize the contrast functions very fast and reliably.

[1]  佐藤 保,et al.  Principal Components , 2021, Encyclopedic Dictionary of Archaeology.

[2]  H. Harman.  Modern factor analysis , 1961 .

[3]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[4]  E. B. Andersen,et al.  Modern factor analysis , 1961 .

[5]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[6]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[7]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[8]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[9]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  Dinh Tuan Pham,et al.  Separation of a mixture of independent sources through a maximum likelihood approach , 1992 .

[12]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[13]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[14]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[15]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[16]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[17]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[18]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[19]  D. Field,et al.  Natural image statistics and efficient coding. , 1996, Network.

[20]  Aapo Hyvärinen,et al.  New Approximations of Differential Entropy for Independent Component Analysis and Projection Pursuit , 1997, NIPS.

[21]  Erkki Oja,et al.  Independent Component Analysis for Identification of Artifacts in Magnetoencephalographic Recordings , 1997, NIPS.

[22]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[23]  R N Vigário,et al.  Extraction of ocular artefacts from EEG using independent component analysis. , 1997, Electroencephalography and clinical neurophysiology.

[24]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[25]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[26]  A. Hyvarinen.  A family of fixed-point algorithms for independent component analysis , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[27]  A. Hyvarinen,et al.  One-unit contrast functions for independent component analysis: a statistical analysis , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[28]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[29]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[30]  Erkki Oja,et al.  Image feature extraction by sparse coding and independent component analysis , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[31]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[32]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..

[33]  Erkki Oja,et al.  Independent Component Analysis in Wave Decomposition of Auditory Evoked Fields , 1998 .

[34]  Erkki Oja,et al.  An Experimental Comparison of Neural ICA Algorithms , 1998 .

[35]  Erkki Oja,et al.  A fast algorithm for estimating overcomplete ICA bases for image windows , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[36]  Ali Mansour,et al.  Blind Separation of Sources , 1999 .

[37]  E. Parzen.  1. Random Variables and Stochastic Processes , 1999 .