Can recent innovations in harmonic analysis `explain' key findings in natural image statistics?

Recently, applied mathematicians have been pursuing the goal of sparse coding of certain mathematical models of images with edges. They have found by mathematical analysis that, instead of wavelets and Fourier methods, sparse coding leads towards new systems: ridgelets and curvelets. These new systems have elements distributed across a range of scales and locations, but also orientations. In fact they have highly direction-specific elements and exhibit increasing numbers of distinct directions as we go to successively finer scales. Meanwhile, researchers in natural scene statistics (NSS) have been attempting to find sparse codes for natural images. The new systems they have found by computational optimization have elements distributed across a range of scales and locations, but also orientations. The new systems are certainly unlike wavelet and Gabor systems, on the one hand because of the multi-orientation and on the other hand because of the multi-scale nature. There is a certain degree of visual resemblance between the findings in the two fields, which suggests the hypothesis that certain important findings in the NSS literature might possibly be explained by the slogan: edges are the dominant features in images, and curvelets are the right tool for representing edges. We consider here certain empirical consequences of this hypothesis, looking at key findings of the NSS literature and conducting studies of curvelet and ridgelet transforms on synthetic and real images, to see if the results are consistent with predictions from this slogan. Our first experiment measures the nonGaussianity of Fourier, wavelet, ridgelet and curvelet coefficients over a database of synthetic and photographic images. Empirically the curvelet coefficients exhibit noticeably higher kurtosis than wavelet, ridgelet, or Fourier coefficients. This is consistent with the hypothesis. Our second experiment studies the inter-scale correlation of wavelet coefficient energies at the same location. We describe a simple experiment showing that presence of edges explains these correlations. We also develop a crude nonlinear `partial correlation' by considering the correlation between wavelet parents and children after a few curvelet coefficients are removed. When we kill the few biggest coefficients of the curvelet transform, much of the correlation between wavelet subbands disappears - consistent with the hypothesis. We suggest implications for future discussions about NSS.

[1]  D. Pollen,et al.  Phase relationships between adjacent simple cells in the visual cortex. , 1981, Science.

[2]  D. Donoho ON MINIMUM ENTROPY DECONVOLUTION , 1981 .

[3]  H. B. Barlow,et al.  What does the eye see best? , 1983, Nature.

[4]  Daniel A. Pollen,et al.  Visual cortical neurons as localized spatial frequency filters , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[6]  Andrew B. Watson,et al.  The cortex transform: rapid computation of simulated neural images , 1987 .

[7]  Eero P. Simoncelli,et al.  Non-separable extensions of quadrature mirror filters to multiple dimensions , 1990, Proc. IEEE.

[8]  Leslie S. Smith,et al.  The principal components of natural images , 1992 .

[9]  Timothy S. Murphy,et al.  Harmonic Analysis: Real-Variable Methods, Orthogonality, and Oscillatory Integrals , 1993 .

[10]  Y. Meyer Book Review: An introduction to wavelets@@@Book Review: Ten lectures on wavelets , 1993 .

[11]  Yves Meyer Wavelets - algorithms & applications , 1993 .

[12]  D. Ruderman The statistics of natural images , 1994 .

[13]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[14]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[15]  C. Fyfe,et al.  Finding compact and sparse-distributed representations of visual images , 1995 .

[16]  Bernd Girod,et al.  Subband Image Coding , 1996 .

[17]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[18]  R W Prager,et al.  Development of low entropy coding in a recurrent network. , 1996, Network.

[19]  Bruno A. Olshausen,et al.  Inferring Sparse, Overcomplete Image Codes Using an Efficient Coding Framework , 1998, NIPS.

[20]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[21]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[22]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[23]  S. Mallat A wavelet tour of signal processing , 1998 .

[24]  E. Candès Harmonic Analysis of Neural Networks , 1999 .

[25]  Eero P. Simoncelli,et al.  Image compression via joint statistical characterization in the wavelet domain , 1999, IEEE Trans. Image Process..

[26]  F. Ricci Book Review: Harmonic analysis real-variable methods, orthogonality, and oscillatory integrals , 1999 .

[27]  E. Candès,et al.  Ridgelets: a key to higher-dimensional intermittency? , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[28]  D. Donoho Wedgelets: nearly minimax estimation of edges , 1999 .

[29]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[30]  Eero P. Simoncelli,et al.  Texture modeling and synthesis using joint statistics of complex wavelet coefficients , 1999 .

[31]  David L. Donoho,et al.  Orthonormal Ridgelets and Linear Singularities , 2000, SIAM J. Math. Anal..

[32]  Larry L. Schumaker,et al.  Curve and Surface Fitting: Saint-Malo 1999 , 2000 .

[33]  D. Donoho Sparse Components of Images and Optimal Atomic Decompositions , 2001 .