Independent Component Analysis of Image Data

The objective of independent component analysis is to represent a set of multidi-mensional measurement vectors in a basis where the components are statistically independent or as independent as possible. This redundancy reduction between components relates independent component analysis to many other theoretical and practical areas of science that are concerned with information, such as information theory, compression, sensory coding and pattern recognition. One of the rst cases where the principle of redundancy reduction was presented was the analysis of the human visual system. Ever since the rst time this idea was brought up, there has existed a need to study the reduction of statistical dependencies in images. A set of new algorithms for independent component analysis has been developed during the 1990's. Some of these have been applied to image data, but no larger investigation has been performed to determine the properties and limitations of the method to solve this problem. In this master's thesis the applicability of independent component analysis to analyze statistical dependencies in images has been explored. This has been done by applying some of the new algorithms to image data. The relevant properties of the resulting components have been studied, as well as their connections with closely connected areas of information science.

[1]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[2]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[3]  Robert J. McEliece,et al.  The Theory of Information and Coding , 1979 .

[4]  C. Fyfe,et al.  Higher Order Cumulant Maximisation using Non-linear Hebbian and Anti-Hebbian Learning for Adaptive Blind Separation of Source Signals , 1996 .

[5]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[6]  David J. Field.  Visual coding, redundancy, and “feature detection” , 1998 .

[7]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[8]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[9]  Terrence J. Sejnowski,et al.  Edges are the Independent Components of Natural Scenes , 1996, NIPS.

[10]  Jerry M. Mendel,et al.  Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications , 1991, Proc. IEEE.

[11]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[12]  C. L. Nikias,et al.  Signal processing with higher-order spectra , 1993, IEEE Signal Processing Magazine.

[13]  J. Karhunen,et al.  A bigradient optimization approach for robust PCA, MCA, and source separation , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[14]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[15]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[16]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[17]  Eric Moreau,et al.  High order contrasts for self-adaptive source separation criteria for complex source separation , 1996 .

[18]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[19]  A. Hyvarinen,et al.  One-unit contrast functions for independent component analysis: a statistical analysis , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[20]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[21]  Ruey-Wen Liu,et al.  General approach to blind source separation , 1996, IEEE Trans. Signal Process..

[22]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[23]  Truong Q. Nguyen,et al.  Wavelets and filter banks , 1996 .

[24]  Ingrid Daubechies,et al.  Where do wavelets come from? A personal point of view , 1996, Proc. IEEE.

[25]  E. Kandel,et al.  Essentials of Neural Science and Behavior , 1996 .

[26]  Jelena Kovacevic,et al.  Wavelets and Subband Coding , 2013, Prentice Hall Signal Processing Series.

[27]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[28]  Shun-ichi Amari,et al.  Multi-Layer Neural Networks with a Local Adaptive Learning Rule for Blind Separation of Source Signa , 1995 .

[29]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[30]  A. Hyvärinen,et al.  Nonlinear Blind Source Separation by Self-Organizing Maps , 1996 .

[31]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[32]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[33]  P. Pajunen,et al.  Blind source separation and tracking using nonlinear PCA criterion: a least-squares approach , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[34]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[35]  Barak A. Pearlmutter,et al.  Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA , 1996, NIPS.

[36]  Colin Fyfe,et al.  Kurtosis extrema and identification of independent components: a neural network approach , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[37]  David J. Field,et al.  Wavelet-Like Receptive Fields Emerge From a Network That Learns Sparse Codes for Natural Images , 1996 .

[38]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[39]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[40]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .

[41]  P. Pajunen,et al.  Hierarchic Nonlinear PCA Algorithms for Neural Blind Source Separation , 1996 .

[42]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[43]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..