A fast algorithm for estimating overcomplete ICA bases for image windows

We introduce a very fast method for estimating over-complete bases of independent components from image data. This is based on the concept of quasi-orthogonality, which means that in a very high-dimensional space, there can be a large, over-complete set of vectors that are almost orthogonal to each other. Thus we may estimate an over-complete basis by using one-unit ICA algorithms and forcing only partial decorrelation between the different independent components. The method can be implemented using a modification of the FastICA algorithm, which leads to a computationally highly efficient method.

[1]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[2]  P. Pajunen,et al.  Blind separation of binary sources with less sensors than sources , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[3]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[4]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[5]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..

[6]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[7]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[8]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[9]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[10]  Erkki Oja,et al.  Image feature extraction by sparse coding and independent component analysis , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[11]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[12]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[13]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[14]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[15]  Bruno A. Olshausen,et al.  Inferring Sparse, Overcomplete Image Codes Using an Efficient Coding Framework , 1998, NIPS.

[16]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[17]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.