On the second order spatiochromatic structure of natural images

We provide a theoretical analysis of some empirical facts about the second order spatiochromatic structure of natural images in color. In particular, we show that two simple assumptions on the covariance matrices of color images yield eigenvectors made by the Kronecker product of Fourier features times the triad given by luminance plus color opponent channels. The first of these assumptions is second order stationarity while the second one is commutativity between color correlation matrices. The validity of these assumptions and the predicted shape of the PCA components of color images are experimentally observed on two large image databases. As a by-product of this experimental study, we also provide novel data to support an exponential decay law of the spatiochromatic covariance between pairs of pixels as a function of their spatial distance.

[1]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[2]  D. Mumford,et al.  Stochastic models for generic images , 2001 .

[3]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2008, Commun. ACM.

[5]  D. Mackay,et al.  Towards an information-flow model of human behaviour. , 1956, British journal of psychology.

[6]  Ayan Chakrabarti,et al.  Statistics of real-world hyperspectral images , 2011, CVPR 2011.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  M. Langer.  Large-scale failures of f -α scaling in natural image spectra , 2000 .

[9]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[10]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.

[11]  D. Tolhurst,et al.  Amplitude spectra of natural images , 1992 .

[12]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Joseph J Atick,et al.  Could information theory provide an ecological theory of sensory processing? , 2011, Network.

[14]  T. W. Lee,et al.  Chromatic structure of natural scenes. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[16]  Daniel L. Ruderman,et al.  Origins of scaling in natural images , 1996, Vision Research.

[17]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[18]  J. Moran,et al.  Sensation and perception , 1980 .

[19]  G. Buchsbaum,et al.  Trichromacy, opponent colours coding and optimum colour information transmission in the retina , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[20]  D. Rajan.  Probability, Random Variables, and Stochastic Processes , 2017 .

[21]  D. Tolhurst,et al.  Spatiochromatic Properties of Natural Images and Human Vision , 2002, Current Biology.

[22]  M. Langer.  Large-scale failures of f(-alpha) scaling in natural image spectra. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[23]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[24]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[25]  Erik Reinhard,et al.  Image Statistics and their Applications in Computer Graphics , 2010, Eurographics.

[26]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[27]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[28]  Zhaoping Li,et al.  Understanding Retinal Color Coding from First Principles , 1992, Neural Computation.

[29]  Joel Pokorny,et al.  Spectral sensitivities of the human cones , 2013 .

[30]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[31]  Andriana Olmos,et al.  A biologically inspired algorithm for the recovery of shading and reflectance images , 2004 .

[32]  Javier Vazquez-Corral,et al.  Color Constancy Algorithms: Psychophysical Evaluation on a New Dataset , 2009 .

[33]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[34]  Gershon Buchsbaum,et al.  A computational model of spatiochromatic image coding in early vision , 1991, J. Vis. Commun. Image Represent..

[35]  Michael Frazier.  An introduction to wavelets through linear algebra , 1999 .

[36]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[37]  D. L. Macadam.  Chromatic adaptation. , 1956, Journal of the Optical Society of America.

[38]  Vijay Balasubramanian,et al.  Natural Images from the Birthplace of the Human Eye , 2011, PloS one.

[39]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[40]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[41]  T. Kanade,et al.  Color information for region segmentation , 1980 .

[42]  Mark D. Fairchild,et al.  Color Appearance Models , 1997, Computer Vision, A Reference Guide.

[43]  D. Foster,et al.  Frequency of metamerism in natural scenes , 2006 .

[44]  Sabine Süsstrunk,et al.  What is the space of spectral sensitivity functions for digital color cameras? , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[45]  Li Zhaoping,et al.  Understanding Vision: Theory, Models, and Data , 2014 .

[46]  C. Enroth-Cugell,et al.  Chapter 9 Visual adaptation and retinal gain controls , 1984 .

[47]  Michael H. Brill,et al.  Color appearance models , 1998 .

[48]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.