Flexible contextual modulation of naturalistic texture perception in peripheral vision

Peripheral vision comprises most of our visual field, and is essential in guiding visual behavior. Its characteristic capabilities and limitations, which distinguish it from foveal vision, have been explained by the most influential theory of peripheral vision as the product of representing the visual input using summary-statistics. Despite its success, this account may provide a limited understanding of peripheral vision, because it neglects processes of perceptual grouping and segmentation. To test this hypothesis, we studied how contextual modulation, namely the modulation of the perception of a stimulus by its surrounds, interacts with segmentation in human peripheral vision. We used naturalistic textures, which are directly related to summary-statistics representations. We show that segmentation cues affect contextual modulation, and that this is not captured by our implementation of the summary-statistics model. We then characterize the effects of different texture statistics on contextual modulation, providing guidance for extending the model, as well as for probing neural mechanisms of peripheral vision.

[1]  Eero P. Simoncelli,et al.  Article Sound Texture Perception via Statistics of the Auditory Periphery: Evidence from Sound Synthesis , 2022 .

[2]  Hans Strasburger Seven myths on crowding and peripheral vision , 2019 .

[3]  Thomas D. Mrsic-Flogel,et al.  Experience-Dependent Specialization of Receptive Field Surround for Selective Coding of Natural Scenes , 2014, Neuron.

[4]  D. Pelli,et al.  Crowding is unlike ordinary masking: distinguishing feature integration from detection. , 2004, Journal of vision.

[5]  B. Treutwein Adaptive psychophysical procedures , 1995, Vision Research.

[6]  鄭素梅,et al.  Nature Publishing Group , 2006 .

[7]  O. Schwartz,et al.  Flexible Gating of Contextual Influences in Natural Vision , 2015, Nature Neuroscience.

[8]  Greg O. Horne,et al.  Controlling low-level image properties: The SHINE toolbox , 2010, Behavior research methods.

[9]  Frouke Hermens,et al.  Release of crowding by pattern completion. , 2015, Journal of vision.

[10]  C. Koch,et al.  Flanker effects in peripheral contrast discrimination—psychophysics and modeling , 2001, Vision Research.

[11]  O. Schwartz,et al.  Specificity and timescales of cortical adaptation as inferences about natural movie statistics , 2016, Journal of vision.

[12]  G. Westheimer,et al.  Global stimulus configuration modulates crowding. , 2009, Journal of vision.

[13]  David Whitney,et al.  Multi-level Crowding and the Paradox of Object Recognition in Clutter , 2018, Current Biology.

[14]  R. Shapley,et al.  Contextual influences on orientation discrimination: binding local and global cues , 2001, Vision Research.

[15]  Adrien Doerig,et al.  Beyond Bouma's window: How to explain global aspects of crowding? , 2019, PLoS Comput. Biol..

[16]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[17]  H. Neumann,et al.  Neural mechanisms of human texture processing: texture boundary detection and visual search. , 2005, Spatial vision.

[18]  Peter Neri,et al.  Object segmentation controls image reconstruction from natural scenes , 2017, PLoS biology.

[19]  Cristina Meinecke,et al.  Texture segmentation: Do the processing units on the saliency map increase with eccentricity? , 2011, Vision Research.

[20]  R. Rosenholtz,et al.  A summary statistic representation in peripheral vision explains visual search. , 2009, Journal of vision.

[21]  M. Young,et al.  Centre‐surround interactions in response to natural scene stimulation in the primary visual cortex , 2005, The European journal of neuroscience.

[22]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[23]  Yoav Tadmor,et al.  The perceived contrast of texture patches embedded in natural images , 2006, Vision Research.

[24]  Thomas S A Wallis,et al.  Image correlates of crowding in natural scenes. , 2011, Journal of vision.

[25]  Gerald Westheimer,et al.  Contrast polarity, chromaticity, and stereoscopic depth modulate contextual interactions in vernier acuity. , 2008, Journal of vision.

[26]  Lynn A. Olzak,et al.  The extraction of natural scene gist in visual crowding , 2018, Scientific Reports.

[27]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[28]  E. Louie,et al.  Holistic crowding: selective interference between configural representations of faces in crowded scenes. , 2007, Journal of vision.

[29]  Yihui Xie,et al.  Dynamic Documents with R and knitr , 2015 .

[30]  Béla Julesz,et al.  Visual Pattern Discrimination , 1962, IRE Trans. Inf. Theory.

[31]  G Westheimer,et al.  The effect of spacing regularity on visual crowding. , 2010, Journal of vision.

[32]  Mary M. Conte,et al.  Textures as Probes of Visual Processing. , 2017, Annual review of vision science.

[33]  Karl R Gegenfurtner,et al.  Dynamics of oculomotor direction discrimination. , 2012, Journal of vision.

[34]  Jonathan D Victor,et al.  Visual processing of informative multipoint correlations arises primarily in V2 , 2015, eLife.

[35]  Lionel Henry,et al.  A Grammar of Data Manipulation [R package dplyr version 1.0.2] , 2020 .

[36]  S. Grossberg,et al.  Texture segregation by visual cortex: Perceptual grouping, attention, and learning , 2007, Vision Research.

[37]  Jonathan D Victor,et al.  The statistics of local motion signals in naturalistic movies. , 2014, Journal of vision.

[38]  K. E. Overvliet,et al.  Perceptual grouping determines haptic contextual modulation , 2016, Vision Research.

[39]  Anna Shafer-Skelton,et al.  Global Ensemble Texture Representations are Critical to Rapid Scene Perception , 2017, Journal of experimental psychology. Human perception and performance.

[40]  A. Logvinenko The geometric structure of color. , 2015, Journal of vision.

[41]  A. Schmid The processing of feature discontinuities for different cue types in primary visual cortex , 2008, Brain Research.

[42]  Eero P. Simoncelli,et al.  Selectivity and tolerance for visual texture in macaque V2 , 2016, Proceedings of the National Academy of Sciences.

[43]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[44]  Daniel Oberfeld,et al.  Sequential Grouping Modulates the Effect of Non-Simultaneous Masking on Auditory Intensity Resolution , 2012, PloS one.

[45]  Daniel J. Thengone,et al.  Perception of second- and third-order orientation signals and their interactions. , 2013, Journal of vision.

[46]  G Chastain,et al.  Confusability and interference between members of parafoveal letter pairs , 1982, Perception & psychophysics.

[47]  M. Morgan,et al.  The Relationship between Search Efficiency and Crowding , 2007, Perception.

[48]  Bosco S. Tjan,et al.  Crowding during restricted and free viewing , 2013, Vision Research.

[49]  Michael A. Cohen,et al.  What is the Bandwidth of Perceptual Experience? , 2016, Trends in Cognitive Sciences.

[50]  Kazunori Morikawa,et al.  Central performance drop in texture segmentation: the role of spatial and temporal factors , 2000, Vision Research.

[51]  D. Levi,et al.  Visual crowding: a fundamental limit on conscious perception and object recognition , 2011, Trends in Cognitive Sciences.

[52]  J. Victor Images, statistics, and textures: implications of triple correlation uniqueness for texture statistics and the Julesz conjecture: comment , 1994 .

[53]  J. Bergen,et al.  Computational Modeling of Visual Texture Segregation , 1991 .

[54]  B Julesz,et al.  On the Limits of Fourier Decompositions in Visual Texture Perception , 1979, Perception.

[55]  Michael H Herzog,et al.  What crowding can tell us about object representations. , 2016, Journal of vision.

[56]  Eero P. Simoncelli,et al.  Metamers of the ventral stream , 2011, Nature Neuroscience.

[57]  Matthias Bethge,et al.  Testing models of peripheral encoding using metamerism in an oddity paradigm. , 2016, Journal of vision.

[58]  R. Rosenholtz Capabilities and Limitations of Peripheral Vision. , 2016, Annual review of vision science.

[59]  D. Levi,et al.  The effect of similarity and duration on spatial interaction in peripheral vision. , 1994, Spatial vision.

[60]  R. van Lier,et al.  Grouping Effects in Flash-Induced Perceptual Fading , 2007, Perception.

[61]  Jonathan S. Cant,et al.  Independent Processing of Form, Colour, and Texture in Object Perception , 2008, Perception.

[62]  Derrick J. Parkhurst,et al.  Texture contrast attracts overt visual attention in natural scenes , 2004, The European journal of neuroscience.

[63]  Zhaoping Li A saliency map in primary visual cortex , 2002, Trends in Cognitive Sciences.

[64]  J. Lund,et al.  Compulsory averaging of crowded orientation signals in human vision , 2001, Nature Neuroscience.

[65]  Eero P. Simoncelli,et al.  Contextual modulation of sensitivity to naturalistic image structure in macaque V2 , 2018, Journal of neurophysiology.

[66]  S. Morad,et al.  Ceramide-orchestrated signalling in cancer cells , 2012, Nature Reviews Cancer.

[67]  Yury Petrov,et al.  Locus of spatial attention determines inward-outward anisotropy in crowding. , 2011, Journal of vision.

[68]  Mj Sinai,et al.  Egocentric Distance Perception in a Virutal Environment Using a Perceptual Matching Task , 1999 .

[69]  Gregory Francis,et al.  Neural Dynamics of Grouping and Segmentation Explain Properties of Visual Crowding , 2017, Psychological review.

[70]  J. Wagemans The Oxford handbook of perceptual organization , 2015 .

[71]  H. Wickham Easily Tidy Data with 'spread()' and 'gather()' Functions , 2016 .

[72]  Krista A. Ehinger,et al.  A general account of peripheral encoding also predicts scene perception performance. , 2016, Journal of vision.

[73]  P. König,et al.  The role of first- and second-order stimulus features for human overt attention , 2007, Perception & psychophysics.

[74]  Z Li,et al.  Contextual influences in V1 as a basis for pop out and asymmetry in visual search. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[75]  Brian D. Ripley,et al.  Modern applied statistics with S, 4th Edition , 2002, Statistics and computing.

[76]  Yury Petrov,et al.  Asymmetries and idiosyncratic hot spots in crowding , 2011, Vision Research.

[77]  J. Dichgans,et al.  Differential effects of central versus peripheral vision on egocentric and exocentric motion perception , 1973, Experimental Brain Research.

[78]  Chris I Baker,et al.  Contributions of low- and high-level properties to neural processing of visual scenes in the human brain , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[79]  D. Heeger,et al.  Center-surround interactions in foveal and peripheral vision , 2000, Vision Research.

[80]  Gouki Okazawa,et al.  Gradual Development of Visual Texture-Selective Properties Between Macaque Areas V2 and V4 , 2016, Cerebral cortex.

[81]  Johan Wagemans,et al.  Spatial arrangement in texture discrimination and texture segregation , 2013, i-Perception.

[82]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[83]  Curtis L Baker,et al.  Higher order image structure enables boundary segmentation in the absence of luminance or contrast cues. , 2014, Journal of vision.

[84]  Lothar Spillmann,et al.  Texture fading correlates with stimulus salience , 2001, Vision Research.

[85]  A. Thielscher,et al.  Texture segmentation in human perception: A combined modeling and fMRI study , 2008, Neuroscience.

[86]  Hans Strasburger,et al.  Source confusion is a major cause of crowding. , 2013, Journal of vision.

[87]  B. Julesz,et al.  Visual discrimination of textures with identical third-order statistics , 1978, Biological Cybernetics.

[88]  Thomas Serre,et al.  Disentangling neural mechanisms for perceptual grouping , 2019, ICLR.

[89]  A. Doerig,et al.  Crowding reveals fundamental differences in local vs. global processing in humans and machines , 2020, Vision Research.

[90]  Cordelia Schmid,et al.  A sparse texture representation using local affine regions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[91]  Gertjan J. Burghouts,et al.  Material-specific adaptation of color invariant features , 2009, Pattern Recognit. Lett..

[92]  Matteo Carandini,et al.  Two Distinct Mechanisms of Suppression in Human Vision , 2005, The Journal of Neuroscience.

[93]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[94]  W. Banks,et al.  Asymmetry of visual interference , 1979, Perception & psychophysics.

[95]  J. Movshon,et al.  Selectivity and spatial distribution of signals from the receptive field surround in macaque V1 neurons. , 2002, Journal of neurophysiology.

[96]  G. Sperling,et al.  The lateral inhibition of perceived contrast is indifferent to on-center/off-center segregation, but specific to orientation , 1993, Vision Research.

[97]  David Robinson,et al.  Convert Statistical Objects into Tidy Tibbles [R package broom version 0.7.1] , 2020 .

[98]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[99]  R. Rosenholtz,et al.  A summary-statistic representation in peripheral vision explains visual crowding. , 2009, Journal of vision.

[100]  Helena X Wang,et al.  Responses to second-order texture modulations undergo surround suppression , 2012, Vision Research.

[101]  Ignacio Serrano-Pedraza,et al.  Moderate acute alcohol intoxication has minimal effect on surround suppression measured with a motion direction discrimination task. , 2015, Journal of vision.

[102]  C. Meinecke,et al.  Spatial distance between target and irrelevant patch modulates detection in a texture segmentation task. , 2009, Spatial vision.

[103]  M. Herzog,et al.  When crowding of crowding leads to uncrowding. , 2013, Journal of vision.

[104]  David Whitney,et al.  Holistic crowding of Mooney faces. , 2009, Journal of vision.

[105]  M. Ishihara,et al.  The functional role of central and peripheral vision in the control of posture. , 2005, Human movement science.

[106]  Josh H. McDermott,et al.  Adaptive and Selective Time Averaging of Auditory Scenes , 2018, Current Biology.

[107]  M. Braunstein The role of central and peripheral vision in postural control duringwalking , 2010 .

[108]  H. Komatsu,et al.  Image statistics underlying natural texture selectivity of neurons in macaque V4 , 2014, Proceedings of the National Academy of Sciences.

[109]  M. Grassi,et al.  Contextual influences in texture-segmentation: Distinct effects from elements along the edge and in the texture-region , 2013, Vision Research.

[110]  Christina Gloeckner,et al.  Modern Applied Statistics With S , 2003 .

[111]  Toni P Saarela,et al.  Size tuning and contextual modulation of backward contrast masking. , 2009, Journal of vision.

[112]  Curtis L. Baker,et al.  Texture sparseness, but not local phase structure, impairs second-order segmentation , 2013, Vision Research.

[113]  Nicolai Petkov,et al.  Contextual modulation as de-texturizer , 2014, Vision Research.

[114]  D. Pelli,et al.  The Bouma law of crowding, revised: critical spacing is equal across parts, not objects. , 2014, Journal of vision.

[115]  Uri Polat,et al.  Space and time in masking and crowding. , 2015, Journal of vision.

[116]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[117]  David J Heeger,et al.  Response Suppression in V1 Agrees with Psychophysics of Surround Masking , 2003, The Journal of Neuroscience.

[118]  Catherine Hindi Attar,et al.  Uniform versus random orientation in fading and filling-in , 2007, Vision Research.

[119]  Ariella V. Popple,et al.  Crowding and surround suppression: not to be confused. , 2007, Journal of vision.

[120]  D. Bates,et al.  Linear Mixed-Effects Models using 'Eigen' and S4 , 2015 .

[121]  Bilge Sayim,et al.  Grouping, pooling, and when bigger is better in visual crowding. , 2012, Journal of vision.

[122]  Michael S. Landy,et al.  Texture analysis and perception , 2013 .

[123]  M. Landy Texture perception , 1996 .

[124]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[125]  P. Green,et al.  Measuring perceived differences in surface texture due to changes in higher order statistics. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[126]  John W. Eaton,et al.  GNU Octave 4.0 : reference manual , 2015 .

[127]  D. Kersten,et al.  Segmentation decreases the magnitude of the tilt illusion. , 2013, Journal of vision.

[128]  J A Solomon,et al.  Texture interactions determine perceived contrast , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[129]  W. Warren,et al.  The role of central and peripheral vision in postural control duringwalking , 1999, Perception & psychophysics.

[130]  Gerald Westheimer,et al.  Grouping of contextual elements that affect vernier thresholds. , 2007, Journal of vision.

[131]  Mary M. Conte,et al.  Variance predicts salience in central sensory processing , 2014, eLife.

[132]  D. Levi Crowding—An essential bottleneck for object recognition: A mini-review , 2008, Vision Research.

[133]  Leon A. Gatys,et al.  Image content is more important than Bouma’s Law for scene metamers , 2018 .

[134]  Eero P. Simoncelli,et al.  A functional and perceptual signature of the second visual area in primates , 2013, Nature Neuroscience.

[135]  T. L. Harrington,et al.  Perception of Orientation of Motion as Affected by Change in Divergence of Texture, Change in Size, and in Velocity , 1985, Perceptual and motor skills.

[136]  R. Tibshirani,et al.  Lasso and Elastic-Net Regularized Generalized Linear Models [R package glmnet version 4.0-2] , 2020 .

[137]  O. Blanke,et al.  Learning to integrate contradictory multisensory self-motion cue pairings. , 2015, Journal of vision.

[138]  Jonathan D. Victor,et al.  Possible functions of contextual modulations and receptive field nonlinearities: Pop-out and texture segmentation , 2014, Vision Research.

[139]  Robert W. Kentridge,et al.  Separate channels for processing form, texture, and color: evidence from FMRI adaptation and visual object agnosia. , 2010, Cerebral cortex.

[140]  Ruth Rosenholtz,et al.  Challenges to pooling models of crowding: Implications for visual mechanisms , 2019, Journal of vision.

[141]  C Meinecke,et al.  Peripheral and foveal segmentation of angle textures , 1994, Perception & psychophysics.

[142]  J. Bergen,et al.  Texture segregation and orientation gradient , 1991, Vision Research.

[143]  D. Bates,et al.  Parsimonious Mixed Models , 2015, 1506.04967.

[144]  Endel Põder,et al.  Effect of colour pop-out on the recognition of letters in crowding conditions , 2007, Psychological research.

[145]  Jonathan S. Cant,et al.  Object Ensemble Processing in Human Anterior-Medial Ventral Visual Cortex , 2012, The Journal of Neuroscience.

[146]  M. Herzog,et al.  Crowding, grouping, and object recognition: A matter of appearance. , 2015, Journal of vision.

[147]  Ken Nakayama,et al.  Brightness perception and filling-in , 1991, Vision Research.