Image feature extraction by sparse coding and independent component analysis

Sparse coding is a method for finding a representation of data in which each of the components of the representation is only rarely significantly active. Such a representation is closely related to the techniques of independent component analysis and blind source separation. In this paper, we investigate the application of sparse coding for image feature extraction. We show how sparse coding can be used to extract wavelet-like features from natural image data. As an application of such a feature extraction scheme, we show how to apply a soft-thresholding operator on the components of sparse coding in order to reduce Gaussian noise. Methods based on sparse coding have the important benefit over wavelet methods that the features are determined solely by the statistical properties of the data, while the wavelet transformation relies heavily on certain abstract mathematical properties that may be only weakly related to the properties of the natural data.

[1]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[2]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[3]  Aapo Hyvärinen,et al.  Denoising of Sensory Data by Maximum Likelihood Estimation of Sparse Components , 1998 .

[4]  Horace Barlow,et al.  What is the computational goal of the neocortex , 1994 .

[5]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[6]  Aapo Hyvärinen,et al.  Independent component analysis in the presence of Gaussian noise by maximizing joint likelihood , 1998, Neurocomputing.

[7]  A. Hyvarinen.  A family of fixed-point algorithms for independent component analysis , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[9]  Hyvarinen.  Sparse code shrinkage: denoising of nongaussian data by maximum likelihood estimation , 1999, Neural computation.

[10]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[11]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[12]  Aapo Hyvärinen,et al.  Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation , 1999, Neural Computation.

[13]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[14]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[15]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..