Independent Component Analysis

The goal of blind source separation (BSS) is to recover independent sources given only sensor observations that are linear mixtures of independent source signals. The term blind indicates that both the source signals and the way the signals were mixed are unknown. Independent Component Analysis (ICA) is a method for solving the blind source separation problem. It is a way to find a linear coordinate system (the unmixing system) such that the resulting signals are as statistically independent from each other as possible. In contrast to correlation-based transformations such as Principal Component Analysis (PCA), ICA not only decorrelates the signals (2nd-order statistics) but also reduces higher-order statistical dependencies.

[1]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[2]  R. Aris.  Vectors, Tensors and the Basic Equations of Fluid Mechanics , 1962 .

[3]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[4]  P. Nunez,et al.  Electric fields of the brain , 1981 .

[5]  S. Liberty,et al.  Linear Systems , 2010, Scientific Parallel Computing.

[6]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[7]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Jerry M. Mendel,et al.  Maximum-Likelihood Deconvolution , 1989 .

[9]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[10]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[11]  Christian Jutten,et al.  Space or time adaptive signal processing by neural network models , 1987 .

[12]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[13]  Ralph Linsker,et al.  An Application of the Principle of Maximum Information Preservation to Linear Systems , 1988, NIPS.

[14]  Eric A. Vittoz,et al.  CMOS Integration of Herault-Jutten Cells for Separation of Sources , 1989, Analog VLSI Implementation of Neural Systems.

[15]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[16]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1995, COLT '90.

[17]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[18]  J J Hopfield,et al.  Olfactory computation and object perception. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[19]  John C. Platt,et al.  Networks for the Separation of Sources that Are Superimposed and Delayed , 1991, NIPS.

[20]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[21]  P. Berg,et al.  Dipole models of eye movements and blinks. , 1991, Electroencephalography and clinical neurophysiology.

[22]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[23]  Esfandiar Sorouchyari,et al.  Blind separation of sources, part III: Stability analysis , 1991, Signal Process..

[24]  Ralph Linsker,et al.  Local Synaptic Learning Rules Suffice to Maximize Mutual Information in a Linear Network , 1992, Neural Computation.

[25]  Dinh Tuan Pham,et al.  Separation of a mixture of independent sources through a maximum likelihood approach , 1992 .

[26]  R. Turner,et al.  Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Andreas G. Andreou,et al.  Current-mode subthreshold MOS implementation of the Herault-Jutten autoadaptive network , 1992 .

[28]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[29]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[30]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[31]  Zoubin Ghahramani,et al.  Solving inverse problems using an EM approach to density estimation , 1993 .

[32]  Meir Feder,et al.  Multi-channel signal separation by decorrelation , 1993, IEEE Trans. Speech Audio Process..

[33]  Joseph J. Atick,et al.  Convergent Algorithm for Sensory Receptive Field Development , 1993, Neural Computation.

[34]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[35]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[36]  Ah Chung Tsoi,et al.  Blind deconvolution of signals using a complex recurrent network , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[37]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[38]  Ehud Weinstein,et al.  Criteria for multichannel signal separation , 1994, IEEE Trans. Signal Process..

[39]  S. Srihari.  Mixture Density Networks , 1994 .

[40]  Andrzej Cichocki,et al.  Robust learning algorithm for blind separation of signals , 1994 .

[41]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[42]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[43]  Terrence J. Sejnowski,et al.  Adaptive separation of mixed broadband sound sources with delays by a beamforming Herault-Jutten network , 1995 .

[44]  C. L. Nikias,et al.  A sliding cost function algorithm for blind deconvolution , 1995, Conference Record of The Twenty-Ninth Asilomar Conference on Signals, Systems and Computers.

[45]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[46]  David Mackay,et al.  Probable networks and plausible predictions - a review of practical Bayesian methods for supervised neural networks , 1995 .

[47]  Shun-ichi Amari,et al.  Information geometry of the EM and em algorithms for neural networks , 1995, Neural Networks.

[48]  Shun-ichi Amari,et al.  Multi-Layer Neural Networks with a Local Adaptive Learning Rule for Blind Separation of Source Signa , 1995 .

[49]  Erkki Oja,et al.  Signal Separation by Nonlinear Hebbian Learning , 1995 .

[50]  Juha Karhunen,et al.  Nonlinear PCA type approaches for source separation and independent component analysis , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[51]  Kiyotoshi Matsuoka,et al.  A neural net for blind separation of nonstationary signals , 1995, Neural Networks.

[52]  Chrysostomos L. Nikias,et al.  Polynomial matrix whitening and application to the multichannel blind deconvolution problem , 1995, Proceedings of MILCOM '95.

[53]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[54]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[55]  G. Deco,et al.  An Information-Theoretic Approach to Neural Computing , 1997, Perspectives in Neural Computing.

[56]  Barak A. Pearlmutter,et al.  Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA , 1996, NIPS.

[57]  Kari Torkkola,et al.  Blind separation of convolved sources based on information maximization , 1996, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop.

[58]  Lizhong Wu,et al.  Multi-effect Decompositions for Financial Data Modeling , 1996, NIPS.

[59]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[60]  Steve Rogers,et al.  Adaptive Filter Theory , 1996 .

[61]  Petteri Pajunen,et al.  Nonlinear Blind Source Separation by Self-Organizing Maps , 1996 .

[62]  Roland Gautier,et al.  Blind separation of convolutive mixtures using second and fourth order moments , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[63]  Barak A. Pearlmutter,et al.  A Context-Sensitive Generalization of ICA , 1996 .

[64]  P. Comon,et al.  Contrasts for multichannel blind deconvolution , 1996, IEEE Signal Processing Letters.

[65]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[66]  Pierre Comon,et al.  Independent component analysis, a survey of some algebraic methods , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[67]  R. Lambert.  Multichannel blind deconvolution: FIR matrix algebra and separation of multipath mixtures , 1996 .

[68]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[69]  Juha Karhunen,et al.  Neural approaches to independent component analysis and source separation , 1996, ESANN.

[70]  Simon J. Godsill,et al.  Multi-channel signal separation , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[71]  T J Sejnowski,et al.  Learning the higher-order structure of a natural sound. , 1996, Network.

[72]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[73]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[74]  Andreas Ziehe,et al.  Adaptive On-line Learning in Changing Environments , 1996, NIPS.

[75]  Erkki Oja,et al.  One-unit Learning Rules for Independent Component Analysis , 1996, NIPS.

[76]  Terrence J. Sejnowski,et al.  Independent Component Analysis of Simulated EEG Using a Three-Shell Spherical Head Model. , 1996 .

[77]  Marian Stewart Bartlett,et al.  Viewpoint Invariant Face Recognition using Independent Component Analysis and Attractor Networks , 1996, NIPS.

[78]  Shun-ichi Amari,et al.  Neural Learning in Structured Parameter Spaces - Natural Riemannian Gradient , 1996, NIPS.

[79]  Aapo Hyvärinen,et al.  A family of fixed-point algorithms for independent component analysis , 1997, ICASSP.

[80]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[81]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[82]  C. Fyfe,et al.  Generalised independent component analysis through unsupervised learning with emergent Bussgang properties , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[83]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[84]  T. Sejnowski,et al.  Estimating alertness from the EEG power spectrum , 1997, IEEE Transactions on Biomedical Engineering.

[85]  J. Binder,et al.  Neuroanatomy of language processing studied with functional MRI. , 1997, Clinical neuroscience.

[86]  H. Yangy,et al.  Information Back-propagation for Blind Separation of Sources in Non-linear Mixture , 1997 .

[87]  Juan K. Lin,et al.  Feature extraction approach to blind source separation , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[88]  Tzyy-Ping Jung,et al.  Extended ICA Removes Artifacts from Electroencephalographic Recordings , 1997, NIPS.

[89]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[90]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[91]  D. Manoach,et al.  Prefrontal cortex fMRI signal changes are correlated with working memory load , 1997, Neuroreport.

[92]  Russell H. Lambert,et al.  Blind separation of multiple speakers in a multipath environment , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[93]  Jack D. Cowan,et al.  Faithful Representation Of Separable Input Distributions , 1997, Neural Computation.

[94]  Juha Karhunen,et al.  On Neural Blind Separation with Noise Suppression and Redundancy Reduction , 1997, Int. J. Neural Syst..

[95]  Reinhold Orglmeister,et al.  Blind source separation of real world signals , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[96]  Reinhold Orglmeister,et al.  Improving the Performance of Infomax Using Statistical Signal Processing Techniques , 1997, ICANN.

[97]  Erkki Oja,et al.  Applications of neural blind separation to signal and image processing , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[98]  Christian Jutten,et al.  Nonlinear source separation: the post-nonlinear mixtures , 1997, ESANN.

[99]  Geoffrey E. Hinton,et al.  Generative models for discovering sparse distributed representations. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[100]  Bruno A. Olshausen,et al.  Inferring Sparse, Overcomplete Image Codes Using an Efficient Coding Framework , 1998, NIPS.

[101]  Shun-ichi Amari,et al.  Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information , 1997, Neural Computation.

[102]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[103]  Colin Fyfe,et al.  Stochastic ICA Contrast Maximisation Using Oja's Nonlinear PCA Algorithm , 1997, Int. J. Neural Syst..

[104]  Shun-ichi Amari,et al.  Stability Analysis Of Adaptive Blind Source Separation , 1997 .

[105]  Frank Ehlers,et al.  Blind separation of convolutive mixtures and an application in automatic speech recognition in a noisy environment , 1997, IEEE Trans. Signal Process..

[106]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[107]  Kari Torkkola.  Blind Separation of Radio Signals in Fading Channels , 1997, NIPS.

[108]  A. Cichocki,et al.  Blind signal deconvolution by spatio-temporal decorrelation and demixing , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[109]  P. Pajunen,et al.  Blind separation of binary sources with less sensors than sources , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[110]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[111]  Paris Smaragdis,et al.  Information theoretic approaches to source separation , 1997 .

[112]  Richard S. J. Frackowiak,et al.  Functional localization of the system for visuospatial attention using positron emission tomography. , 1997, Brain : a journal of neurology.

[113]  Colin Fyfe,et al.  An extended exploratory projection pursuit network with linear and nonlinear anti-hebbian lateral connections applied to the cocktail party problem , 1997, Neural Networks.

[114]  Shun-ichi Amari,et al.  Neural Network Models for Blind Separation of Time Delayed and Convolved Signals , 1997 .

[115]  Terrence J. Sejnowski,et al.  Learning Nonlinear Overcomplete Representations for Efficient Coding , 1997, NIPS.

[116]  Eric A. Wan,et al.  Neural speech enhancement using dual extended Kalman filtering , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[117]  Mark Girolami,et al.  Extraction of independent signal sources using a deflationary exploratory projection pursuit network , 1997 .

[118]  S. Amari,et al.  Multichannel blind separation and deconvolution of sources with arbitrary distributions , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[119]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[120]  Martin J. McKeown,et al.  Removing electroencephalographic artifacts: comparison between ICA and PCA , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[121]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[122]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic and Event-Related Potential Data , 1998 .

[123]  J. Nadal.  Blind Source Separation with Time Dependent Mixtures Submitted to Signal Processing , 1998 .

[124]  Terrence J. Sejnowski,et al.  Coding Time-Varying Signals Using Sparse, Shift-Invariant Representations , 1998, NIPS.

[125]  Terrence J. Sejnowski,et al.  Unsupervised Classification with Non-Gaussian Mixture Models Using ICA , 1998, NIPS.

[126]  S Makeig,et al.  Analysis of fMRI data by blind separation into independent spatial components , 1998, Human brain mapping.

[127]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[128]  Hagai Attias,et al.  Blind Source Separation and Deconvolution: The Dynamic Component Analysis Algorithm , 1998, Neural Computation.

[129]  Mark A. Girolami,et al.  An Alternative Perspective on Adaptive Independent Component Analysis Algorithms , 1998, Neural Computation.

[130]  Paul A. Viola,et al.  Restructuring Sparse High Dimensional Data for Effective Retrieval , 1998, NIPS.

[131]  Andreas Ziehe,et al.  Combining time-delayed decorrelation and ICA: towards solving the cocktail party problem , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[132]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[133]  Shun-ichi Amari,et al.  Superefficiency in blind source separation , 1999, IEEE Trans. Signal Process..

[134]  Jürgen Schmidhuber,et al.  Feature Extraction Through LOCOCODE , 1999, Neural Computation.

[135]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[136]  Terrence J. Sejnowski,et al.  Blind source separation of more sources than mixtures using overcomplete representations , 1999, IEEE Signal Processing Letters.

[137]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[138]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[139]  Rekha Govil,et al.  Neural Networks in Signal Processing , 2000 .