Cerebral Cortex doi:10.1093/cercor/bhr357 Higher Level Visual Cortex Represents Retinotopic, Not Spatiotopic, Object Location

The crux of vision is to identify objects and determine their locations in the environment. Although initial visual representations are necessarily retinotopic (eye centered), interaction with the real world requires spatiotopic (absolute) location information. We asked whether higher level human visual cortex-important for stable object recognition and action-contains information about retinotopic and/or spatiotopic object position. Using functional magnetic resonance imaging multivariate pattern analysis techniques, we found information about both object category and object location in each of the ventral, dorsal, and early visual regions tested, replicating previous reports. By manipulating fixation position and stimulus position, we then tested whether these location representations were retinotopic or spatiotopic. Crucially, all location information was purely retinotopic. This pattern persisted when location information was irrelevant to the task, and even when spatiotopic (not retinotopic) stimulus position was explicitly emphasized. We also conducted a "searchlight" analysis across our entire scanned volume to explore additional cortex but again found predominantly retinotopic representations. The lack of explicit spatiotopic representations suggests that spatiotopic object position may instead be computed indirectly and continually reconstructed with each eye movement. Thus, despite our subjective impression that visual information is spatiotopic, even in higher level visual cortex, object location continues to be represented in retinotopic coordinates.

[1]  L E Mays,et al.  Saccades are spatially, not retinocentrically, coded. , 1980, Science.

[2]  I. Biederman,et al.  Neural encoding of relative position. , 2011, Journal of experimental psychology. Human perception and performance.

[3]  J. Theeuwes,et al.  Gradual Remapping Results in Early Retinotopic and Late Spatiotopic Inhibition of Return , 2010, Psychological science.

[4]  John H. R. Maunsell,et al.  Shape selectivity in primate lateral intraparietal cortex , 1998, Nature.

[5]  Marc Joliot,et al.  Eye position‐dependent activity in the primary visual area as revealed by fMRI , 2007, Human brain mapping.

[6]  E. DeYoe,et al.  Mapping striate and extrastriate visual areas in human cerebral cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[7]  C. Colby,et al.  Trans-saccadic perception , 2008, Trends in Cognitive Sciences.

[8]  Jeremy Freeman,et al.  Orientation Decoding Depends on Maps, Not Columns , 2011, The Journal of Neuroscience.

[9]  C. Genovese,et al.  Spatial Updating in Human Parietal Cortex , 2003, Neuron.

[10]  David C. Burr,et al.  Spatiotopic Coding of BOLD Signal in Human Visual Cortex Depends on Spatial Attention , 2011, PloS one.

[11]  R. M. Siegel,et al.  Maps of Visual Space in Human Occipital Cortex Are Retinotopic, Not Spatiotopic , 2008, The Journal of Neuroscience.

[12]  Francisco Pereira,et al.  Information mapping with pattern classifiers: A comparative study , 2011, NeuroImage.

[13]  A. O. Dick,et al.  Effect of eye movements on backward masking and perceived location , 1973 .

[14]  Lotfi B Merabet,et al.  Visual Topography of Human Intraparietal Sulcus , 2007, The Journal of Neuroscience.

[15]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[16]  J. Pratt,et al.  Oculocentric coding of inhibited eye movements to recently attended locations. , 2000, Journal of experimental psychology. Human perception and performance.

[17]  S. R. Lehky,et al.  Comparison of shape encoding in primate dorsal and ventral visual pathways. , 2007, Journal of neurophysiology.

[18]  Adrian T. Lee,et al.  fMRI of human visual cortex , 1994, Nature.

[19]  Leslie G. Ungerleider Two cortical visual systems , 1982 .

[20]  K. Grill-Spector,et al.  fMR-adaptation: a tool for studying the functional properties of human cortical neurons. , 2001, Acta psychologica.

[21]  Russell A. Epstein,et al.  Position selectivity in scene- and object-responsive occipitotemporal regions. , 2007, Journal of neurophysiology.

[22]  D. B. Bender,et al.  Visual properties of neurons in inferotemporal cortex of the Macaque. , 1972, Journal of neurophysiology.

[23]  M Concetta Morrone,et al.  Neural mechanisms for timing visual events are spatially selective in real-world coordinates , 2007, Nature Neuroscience.

[24]  G. Glover,et al.  Retinotopic organization in human visual cortex and the spatial precision of functional MRI. , 1997, Cerebral cortex.

[25]  J. Malpeli,et al.  Responses of neurons in primary visual cortex are modulated by eye position. , 1993, Journal of neurophysiology.

[26]  Richard A. Andersen,et al.  Separate body- and world-referenced representations of visual space in parietal cortex , 1998, Nature.

[27]  E. Zohary,et al.  Rapid Formation of Spatiotopic Representations As Revealed by Inhibition of Return , 2010, The Journal of Neuroscience.

[28]  Sabine Kastner,et al.  Representation of Eye Movements and Stimulus Motion in Topographically Organized Areas of Human Posterior Parietal Cortex , 2008, The Journal of Neuroscience.

[29]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Anitha Pasupathy,et al.  Transformation of shape information in the ventral pathway , 2007, Current Opinion in Neurobiology.

[31]  P. Cavanagh,et al.  Visual stability based on remapping of attention pointers , 2010, Trends in Cognitive Sciences.

[32]  D. Heeger,et al.  Topographic organization for delayed saccades in human posterior parietal cortex. , 2005, Journal of neurophysiology.

[33]  G. Orban,et al.  Coding of Shape and Position in Macaque Lateral Intraparietal Area , 2008, The Journal of Neuroscience.

[34]  Frank Bremmer,et al.  Eye position effects in macaque area V4 , 2000, Neuroreport.

[35]  D. Heeger,et al.  Two Retinotopic Visual Areas in Human Lateral Occipital Cortex , 2006, The Journal of Neuroscience.

[36]  N. Kanwisher,et al.  The Human Body , 2001 .

[37]  Ehud Zohary,et al.  Beyond retinotopic mapping: the spatial representation of objects in the human lateral occipital complex. , 2007, Cerebral cortex.

[38]  Marvin M Chun,et al.  Eye movements help link different views in scene-selective cortex. , 2011, Cerebral cortex.

[39]  J R Duhamel,et al.  The updating of the representation of visual space in parietal cortex by intended eye movements. , 1992, Science.

[40]  C. Galletti,et al.  Gaze-dependent visual neurons in area V3A of monkey prestriate cortex , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[41]  Julie D. Golomb,et al.  Attentional Facilitation throughout Human Visual Cortex Lingers in Retinotopic Coordinates after Eye Movements , 2010, The Journal of Neuroscience.

[42]  Minami Ito,et al.  Size and position invariance of neuronal responses in monkey inferotemporal cortex. , 1995, Journal of neurophysiology.

[43]  D. Melcher Predictive remapping of visual features precedes saccadic eye movements , 2007, Nature Neuroscience.

[44]  F. Bremmer,et al.  Spatial invariance of visual receptive fields in parietal cortex neurons , 1997, Nature.

[45]  Dwight J. Kravitz,et al.  High-level visual object representations are constrained by position. , 2010, Cerebral cortex.

[46]  L. Fogassi,et al.  Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[47]  E. Zohary,et al.  Pattern matching is assessed in retinotopic coordinates. , 2009, Journal of vision.

[48]  Julie D. Golomb,et al.  The Native Coordinate System of Spatial Attention Is Retinotopic , 2008, The Journal of Neuroscience.

[49]  P. Cavanagh,et al.  The gender-specific face aftereffect is based in retinotopic not spatiotopic coordinates across several natural image transformations. , 2009, Journal of vision.

[50]  D. Heeger,et al.  Retinotopy and Functional Subdivision of Human Areas MT and MST , 2002, The Journal of Neuroscience.

[51]  S. Celebrini,et al.  Gaze direction controls response gain in primary visual-cortex neurons , 1999, Nature.

[52]  N. Kanwisher,et al.  A Preference for Contralateral Stimuli in Human Object- and Face-Selective Cortex , 2007, PloS one.

[53]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[54]  Christian F. Doeller,et al.  Lateralized human hippocampal activity predicts navigation based on sequence or place memory , 2010, Proceedings of the National Academy of Sciences.

[55]  Richard A. Andersen,et al.  Coordinate transformations in the representation of spatial information , 1993, Current Opinion in Neurobiology.

[56]  David Melcher,et al.  Spatiotopic temporal integration of visual motion across saccadic eye movements , 2003, Nature Neuroscience.

[57]  S. Gerber,et al.  Unsupervised Natural Experience Rapidly Alters Invariant Object Representation in Visual Cortex , 2008 .

[58]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[59]  Bruno B Averbeck,et al.  Neural Ensemble Decoding Reveals a Correlate of Viewer- to Object-Centered Spatial Transformation in Monkey Parietal Cortex , 2008, The Journal of Neuroscience.

[60]  R. Wurtz Neuronal mechanisms of visual stability , 2008, Vision Research.

[61]  Alex R. Wade,et al.  Visual field maps and stimulus selectivity in human ventral occipital cortex , 2005, Nature Neuroscience.

[62]  B. Butler,et al.  Spatiotopic and retinotopic components of iconic memory , 1987, Psychological research.

[63]  J W Belliveau,et al.  Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. , 1995, Science.

[64]  Hans P. Op de Beeck,et al.  Probing the mysterious underpinnings of multi-voxel fMRI analyses , 2010, NeuroImage.

[65]  C. Gross,et al.  Coding of visual space by premotor neurons. , 1994, Science.

[66]  Arne D. Ekstrom,et al.  Cellular networks underlying human spatial navigation , 2003, Nature.

[67]  Andrew T. Smith,et al.  Representation of eye position in the human parietal cortex. , 2010, Journal of neurophysiology.

[68]  N. Kanwisher,et al.  Only some spatial patterns of fMRI response are read out in task performance , 2007, Nature Neuroscience.

[69]  Jefferson E. Roy,et al.  Representation of Multiple, Independent Categories in the Primate Prefrontal Cortex , 2010, Neuron.

[70]  Benjamin D. Singer,et al.  Retinotopic Organization of Human Ventral Visual Cortex , 2009, The Journal of Neuroscience.

[71]  K. Hoffmann,et al.  Eye position effects in monkey cortex. II. Pursuit- and fixation-related activity in posterior parietal areas LIP and 7A. , 1997, Journal of neurophysiology.

[72]  M. Sereno,et al.  Mapping of Contralateral Space in Retinotopic Coordinates by a Parietal Cortical Area in Humans , 2001, Science.

[73]  Avishai Henik,et al.  Parietal Lobe Lesions Disrupt Saccadic Remapping of Inhibitory Location Tagging , 2004, Journal of Cognitive Neuroscience.

[74]  P. P. Battaglini,et al.  Parietal neurons encoding spatial locations in craniotopic coordinates , 2004, Experimental Brain Research.

[75]  E. Spelke,et al.  Updating egocentric representations in human navigation , 2000, Cognition.

[76]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[77]  Patrick Cavanagh,et al.  The reference frame of the motion aftereffect is retinotopic. , 2009, Journal of vision.

[78]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[79]  K. Hoffmann,et al.  Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. , 1997, Journal of neurophysiology.

[80]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[81]  K. Guo,et al.  Eye position‐dependent activation of neurones in striate cortex of macaque , 1997, Neuroreport.

[82]  Nancy Kanwisher,et al.  The distribution of category and location information across object-selective regions in human visual cortex , 2008, Proceedings of the National Academy of Sciences.

[83]  J. Bisley,et al.  Psychophysical evidence for spatiotopic processing in area MT in a short-term memory for motion task. , 2009, Journal of neurophysiology.

[84]  R. Andersen,et al.  Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[85]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[86]  D. E. Irwin,et al.  Perceiving an integrated visual world , 1993 .

[87]  R. Mansfield,et al.  Analysis of visual behavior , 1982 .

[88]  Mark H Johnson,et al.  Body-centered representations for visually-guided action emerge during early infancy , 1997, Cognition.

[89]  R. M. Siegel,et al.  Encoding of spatial location by posterior parietal neurons. , 1985, Science.

[90]  Christos Constantinidis,et al.  Effects of task and coordinate frame of attention in area 7a of the primate posterior parietal cortex. , 2011, Journal of vision.

[91]  Yale E. Cohen,et al.  A common reference frame for movement plans in the posterior parietal cortex , 2002, Nature Reviews Neuroscience.

[92]  A. Zador,et al.  Neural representation and the cortical code. , 2000, Annual review of neuroscience.

[93]  R. C. Emerson,et al.  Paralysis of the awake human: Visual perceptions , 1976, Vision Research.

[94]  J. Maunsell,et al.  Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object retinal position. , 2003, Journal of neurophysiology.

[95]  David D. Cox,et al.  Untangling invariant object recognition , 2007, Trends in Cognitive Sciences.

[96]  Vincent P. Ferrera,et al.  Visual Remapping by Vector Subtraction: Analysis of Multiplicative Gain Field Models , 2007, Neural Computation.

[97]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[98]  Julie D. Golomb,et al.  Binding of location and color in retinotopic, not spatiotopic, coordinates , 2011 .

[99]  C. Genovese,et al.  Remapping in human visual cortex. , 2007, Journal of neurophysiology.

[100]  R. Andersen,et al.  Multimodal representation of space in the posterior parietal cortex and its use in planning movements. , 1997, Annual review of neuroscience.

[101]  M. Sereno,et al.  A human parietal face area contains aligned head-centered visual and tactile maps , 2006, Nature Neuroscience.

[102]  Ehud Zohary,et al.  Multiple Reference Frames for Saccadic Planning in the Human Parietal Cortex , 2011, The Journal of Neuroscience.

[103]  D. Melcher Spatiotopic Transfer of Visual-Form Adaptation across Saccadic Eye Movements , 2005, Current Biology.

[104]  Hinze Hogendoorn,et al.  Spatial coding and invariance in object-selective cortex , 2011, Cortex.

[105]  David Whitney,et al.  The Emergence of Perceived Position in the Visual System , 2011, Journal of Cognitive Neuroscience.

[106]  Markus Lappe,et al.  The Peri-Saccadic Perception of Objects and Space , 2008, PLoS Comput. Biol..

[107]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[108]  D. Burr,et al.  Spatiotopic selectivity of BOLD responses to visual motion in human area MT , 2007, Nature Neuroscience.

[109]  Avishai Henik,et al.  Hemispheric Asymmetry in the Remapping and Maintenance of Visual Saliency Maps: A TMS Study , 2010, Journal of Cognitive Neuroscience.

[110]  Laure Pisella,et al.  The contribution of spatial remapping impairments to unilateral visual neglect , 2004, Neuroscience & Biobehavioral Reviews.

[111]  D. Heeger,et al.  Topographic maps of visual spatial attention in human parietal cortex. , 2005, Journal of neurophysiology.

[112]  K. Grill-Spector,et al.  Relating retinotopic and object-selective responses in human lateral occipital cortex. , 2008, Journal of neurophysiology.

[113]  T. Vilis,et al.  Gaze-Centered Updating of Visual Space in Human Parietal Cortex , 2003, The Journal of Neuroscience.

[114]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[115]  Tutis Vilis,et al.  Eye position signals modulate early dorsal and ventral visual areas. , 2002, Cerebral cortex.