Efficient coding of natural scene statistics predicts discrimination thresholds for grayscale textures

Previously, in [1], we showed that when sampling is limiting, the efficient coding principle leads to a “variance is salience” hypothesis, and that this hypothesis accounts for visual sensitivity to binary image statistics. Here, using extensive new psychophysical data and image analysis, we show that this hypothesis accounts for visual sensitivity to a large set of grayscale image statistics at a striking level of detail, and also identify the limits of the prediction. We define a 66-dimensional space of local grayscale light-intensity correlations, and measure the relevance of each direction to natural scenes. The “variance is salience” hypothesis predicts that two-point correlations are most salient, and predicts their relative salience. We tested these predictions in a texture-segregation task using un-natural, synthetic textures. As predicted, correlations beyond second order are not salient, and predicted thresholds for over 300 second-order correlations match psychophysical thresholds closely (median fractional error < 0.13).

[1]  Mary M. Conte,et al.  Interaction of luminance and higher-order statistics in texture discrimination , 2005, Vision Research.

[2]  Michael S. Lewicki,et al.  Efficient auditory coding , 2006, Nature.

[3]  J. H. Hateren,et al.  Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation , 1992, Journal of Comparative Physiology A.

[4]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[5]  J. Alonso,et al.  Neuronal and Perceptual Differences in the Temporal Processing of Darks and Lights , 2014, Neuron.

[6]  Daniel J. Thengone,et al.  Perception of second- and third-order orientation signals and their interactions. , 2013, Journal of vision.

[7]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[8]  Jonathan D. Victor,et al.  Perceptual salience of fourth-order visual textures and natural scene statistics , 2013 .

[9]  D. K. Pickard,et al.  Unilateral Markov fields , 1980, Advances in Applied Probability.

[10]  Vijay Balasubramanian,et al.  Design of a Trichromatic Cone Array , 2010, PLoS Comput. Biol..

[11]  Charles P. Ratliff,et al.  Retina is structured to process an excess of darkness in natural scenes , 2010, Proceedings of the National Academy of Sciences.

[12]  J. Victor,et al.  Temporal Encoding of Spatial Information during Active Visual Fixation , 2012, Current Biology.

[13]  J. Victor,et al.  The unsteady eye: an information-processing stage, not a bug , 2015, Trends in Neurosciences.

[14]  Gasper Tkacik,et al.  Local statistics in natural scenes predict the saliency of synthetic textures , 2010, Proceedings of the National Academy of Sciences.

[15]  Vijay Balasubramanian,et al.  The complexity dividend: when sophisticated inference matters , 2019 .

[16]  Daniel J. Thengone,et al.  A perceptual space of local image statistics , 2015, Vision Research.

[17]  Charles P. Ratliff,et al.  Design of a Neuronal Array , 2008, The Journal of Neuroscience.

[18]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[19]  Simona Cocco,et al.  Adaptation of olfactory receptor abundances for efficient coding , 2019, eLife.

[20]  David Pfau,et al.  Dead leaves and the dirty ground: low-level image statistics in transmissive and occlusive imaging environments. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[21]  M. Landy,et al.  A visual mechanism tuned to black , 2004, Vision Research.

[22]  Chun-I Yeh,et al.  On and off domains of geniculate afferents in cat primary visual cortex , 2008, Nature Neuroscience.

[23]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[24]  Mary M. Conte,et al.  Local image statistics: maximum-entropy constructions and perceptual salience. , 2012, Journal of the Optical Society of America. A, Optics, image science, and vision.

[25]  Michael S. Lewicki,et al.  Characterization of Minimum Error Linear Coding with Sensory and Neural Noise , 2011, Neural Computation.

[26]  Vijay Balasubramanian,et al.  Transformation of Stimulus Correlations by the Retina , 2013, PLoS Comput. Biol..

[27]  Nicole L. Carlson,et al.  Sparse Codes for Speech Predict Spectrotemporal Receptive Fields in the Inferior Colliculus , 2012, PLoS Comput. Biol..

[28]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[29]  Jonathan D. Victor,et al.  Two representations of a high-dimensional perceptual space , 2017, Vision Research.

[30]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[31]  Vijay Balasubramanian,et al.  Natural Images from the Birthplace of the Human Eye , 2011, PloS one.

[32]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[33]  V Zemon,et al.  Asymmetries in ON and OFF visual pathways of humans revealed using contrast-evoked cortical potentials , 1988, Visual Neuroscience.

[34]  Peter Sterling,et al.  Principles of Neural Design , 2015 .

[35]  Jason S. Prentice,et al.  A principle of economy predicts the functional architecture of grid cells , 2013, eLife.

[36]  Michael S. Lewicki,et al.  Efficient coding of natural sounds , 2002, Nature Neuroscience.

[37]  Adrienne L. Fairhall,et al.  Efficiency and ambiguity in an adaptive neural code , 2001, Nature.

[38]  Mary M. Conte,et al.  Variance predicts salience in central sensory processing , 2014, eLife.

[39]  Qasim Zaidi,et al.  Neuronal nonlinearity explains greater visual spatial resolution for darks than lights , 2014, Proceedings of the National Academy of Sciences.

[40]  T. Sharpee,et al.  Predictable irregularities in retinal receptive fields , 2009, Proceedings of the National Academy of Sciences.

[41]  S. Laughlin.  A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.