Estimating Overcomplete Independent Component Bases for Image Windows

Estimating overcomplete ICA bases for image windows is a difficult problem. Most algorithms require the estimation of values of the independent components which leads to computationally heavy procedures. Here we first review the existing methods, and then introduce two new algorithms that estimate an approximate overcomplete basis quite fast in a high-dimensional space. The first algorithm is based on the prior assumption that the basis vectors are randomly distributed in the space, and therefore close to orthogonal. The second replaces the conventional orthogonalization procedure by a transformation of the marginal density to gaussian.

[1]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[2]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[3]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[4]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[5]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[6]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[7]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[8]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[9]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[10]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[11]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[12]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[13]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[14]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[15]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[16]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[17]  P. Pajunen,et al.  Blind separation of binary sources with less sensors than sources , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[18]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[19]  Samuel Kaski,et al.  Dimensionality reduction by random mapping: fast similarity computation for clustering , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[20]  Aapo Hyvärinen,et al.  Independent component analysis in the presence of Gaussian noise by maximizing joint likelihood , 1998, Neurocomputing.

[21]  Eero P. Simoncelli,et al.  Modeling Surround Suppression in V1 Neurons with a Statistically Derived Normalization Model , 1998, NIPS.

[22]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..

[23]  K. Jarrod Millman,et al.  Learning Sparse Codes with a Mixture-of-Gaussians Prior , 1999, NIPS.

[24]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[25]  Erkki Oja,et al.  A fast algorithm for estimating overcomplete ICA bases for image windows , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[26]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[27]  Gerhard Krieger,et al.  Nonlinear neurons and higher-order statistics: new approaches to human vision and digital image processing , 1999, Electronic Imaging.

[28]  Bruno A. Olshausen,et al.  PROBABILISTIC FRAMEWORK FOR THE ADAPTATION AND COMPARISON OF IMAGE CODES , 1999 .

[29]  Terrence J. Sejnowski,et al.  Blind source separation of more sources than mixtures using overcomplete representations , 1999, IEEE Signal Processing Letters.

[30]  Aapo Hyvärinen,et al.  Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation , 1999, Neural Computation.

[31]  Antti Honkela,et al.  Bayesian Non-Linear Independent Component Analysis by Multi-Layer Perceptrons , 2000 .

[32]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[33]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[34]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[35]  Aapo Hyvärinen,et al.  Two Methods for Estimating Overcomplete Independent Component Bases , 2001 .

[36]  E. Oja,et al.  Independent Component Analysis , 2013 .

[37]  Aapo Hyvärinen,et al.  Topographic Independent Component Analysis , 2001, Neural Computation.

[38]  M. Inki,et al.  Two approaches to estimation of overcomplete independent component bases , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[39]  Arthur E. C. Pece,et al.  The Problem of Sparse Image Coding , 2002, Journal of Mathematical Imaging and Vision.