Cortical feedback to V1 and V2 contains unique information about high-level scene structure

Early visual cortical neurons receive highly selective feedforward input, which is amplified or disamplified by contextual feedback and lateral connections. A significant challenge for systems neuroscience is to measure the feature space that drives these feedback channels. We occluded visual scenes and measured non-feedforward stimulated subregions of V1 and V2 using fMRI and multi-voxel pattern analyses. We found that response patterns in these subregions contain two high-level scene features, category and depth information. Responses in non-feedforward stimulated V1 and V2 differed from each other, suggesting that feedback to these two areas has unique information content. Further, we reveal that computational models of visual processing inadequately describe early visual cortex because they do not account for the brain's internal modelling of the world.

[1]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[2]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[3]  Lawrence C. Sincich,et al.  The circuitry of V1 and V2: integration of color, form, and motion. , 2005, Annual review of neuroscience.

[4]  Jack L. Gallant,et al.  A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes , 2015, NeuroImage.

[5]  Brian A. Wandell,et al.  Population receptive field estimates in human visual cortex , 2008, NeuroImage.

[6]  J. Budd.  Extrastriate feedback to primary visual cortex in primates: a quantitative analysis of connectivity , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[7]  Victor A. F. Lamme,et al.  Spatially Pooled Contrast Responses Predict Neural and Perceptual Similarity of Naturalistic Image Categories , 2012, PLoS Comput. Biol..

[8]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[9]  Eero P. Simoncelli,et al.  A functional and perceptual signature of the second visual area in primates , 2013, Nature Neuroscience.

[10]  Nikola T. Markov,et al.  A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex , 2012, Cerebral cortex.

[11]  D. Surmeier,et al.  Grouping of image fragments in primary visual cortex , 2022 .

[12]  Lars Muckli,et al.  Decoding Sound and Imagery Content in Early Visual Cortex , 2014, Current Biology.

[13]  Antonio Torralba,et al.  Depth Estimation from Image Structure , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Li Su,et al.  A Toolbox for Representational Similarity Analysis , 2014, PLoS Comput. Biol..

[15]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[16]  Fraser W. Smith,et al.  Nonstimulated early visual areas carry information about surrounding context , 2010, Proceedings of the National Academy of Sciences.

[17]  Caspar M. Schwiedrzik,et al.  Stimulus Predictability Reduces Responses in Primary Visual Cortex , 2010, The Journal of Neuroscience.

[18]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[19]  Jonathan Winawer,et al.  A Two-Stage Cascade Model of BOLD Responses in Human Visual Cortex , 2013, PLoS Comput. Biol..

[20]  A. Clark,et al.  On the functions, mechanisms, and malfunctions of intracortical contextual modulation , 2015, Neuroscience & Biobehavioral Reviews.

[21]  Lars Muckli,et al.  Network interactions: non-geniculate input to V1 , 2013, Current Opinion in Neurobiology.

[22]  Lucy S. Petro,et al.  Contextual Feedback to Superficial Layers of V1 , 2015, Current Biology.

[23]  Thomas Serre,et al.  Neuronal Synchrony in Complex-Valued Deep Networks , 2013, ICLR.

[24]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[25]  Lars Muckli,et al.  Contributions of cortical feedback to sensory processing in primary visual cortex , 2014, Front. Psychol..

[26]  C. Gilbert,et al.  Top-down influences on visual processing , 2013, Nature Reviews Neuroscience.

[27]  D. Mumford,et al.  The role of the primary visual cortex in higher level vision , 1998, Vision Research.

[28]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[29]  Sennay Ghebreab,et al.  From Image Statistics to Scene Gist: Evoked Neural Activity Reveals Transition from Low-Level Natural Image Structure to Scene Category , 2013, The Journal of Neuroscience.

[30]  N. Kanwisher,et al.  Feedback of pVisual Object Information to Foveal Retinotopic Cortex , 2008, Nature Neuroscience.

[31]  Matteo Carandini,et al.  Faculty Opinions recommendation of A functional and perceptual signature of the second visual area in primates. , 2013 .

[32]  Philippe G Schyns,et al.  Decoding face categories in diagnostic subregions of primary visual cortex , 2013, The European journal of neuroscience.

[33]  Lars Muckli,et al.  Primary Visual Cortex Activity along the Apparent-Motion Trace Reflects Illusory Perception , 2005, PLoS biology.

[34]  Jean-Baptiste Poline,et al.  Inverse retinotopy: Inferring the visual content of images from brain activation patterns , 2006, NeuroImage.

[35]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[36]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.