Stereoscopically Observing Manipulative Actions

The purpose of this study was to investigate the contribution of stereopsis to the processing of observed manipulative actions. To this end, we first combined the factors “stimulus type” (action, static control, and dynamic control), “stereopsis” (present, absent) and “viewpoint” (frontal, lateral) into a single design. Four sites in premotor, retro-insular (2) and parietal cortex operated specifically when actions were viewed stereoscopically and frontally. A second experiment clarified that the stereo-action-specific regions were driven by actions moving out of the frontoparallel plane, an effect amplified by frontal viewing in premotor cortex. Analysis of single voxels and their discriminatory power showed that the representation of action in the stereo-action-specific areas was more accurate when stereopsis was active. Further analyses showed that the 4 stereo-action-specific sites form a closed network converging onto the premotor node, which connects to parietal and occipitotemporal regions outside the network. Several of the specific sites are known to process vestibular signals, suggesting that the network combines observed actions in peripersonal space with gravitational signals. These findings have wider implications for the function of premotor cortex and the role of stereopsis in human behavior.

[1]  F. Mast,et al.  The human vestibular cortex revealed by coordinate-based activation likelihood estimation meta-analysis , 2012, Neuroscience.

[2]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[3]  Guy A. Orban,et al.  Mapping the parietal cortex of human and non-human primates , 2006, Neuropsychologia.

[4]  Miranda Scolari,et al.  Estimating the influence of attention on population codes in human visual cortex using voxel-based tuning functions , 2009, NeuroImage.

[5]  J Randall Flanagan,et al.  Where One Hand Meets the Other: Limb-Specific and Action-Dependent Movement Plans Decoded from Preparatory Signals in Single Human Frontoparietal Brain Areas , 2013, The Journal of Neuroscience.

[6]  Velia Cardin,et al.  Sensitivity of human visual and vestibular cortical regions to egomotion-compatible visual stimulation. , 2010, Cerebral cortex.

[7]  S Dehaene,et al.  Spatially invariant coding of numerical information in functionally defined subregions of human parietal cortex. , 2015, Cerebral cortex.

[8]  Karl J. Friston,et al.  Generalisability, Random Effects & Population Inference , 1998, NeuroImage.

[9]  Dora E Angelaki,et al.  Convergence of Vestibular and Visual Self-Motion Signals in an Area of the Posterior Sylvian Fissure , 2011, The Journal of Neuroscience.

[10]  Valeria I. Petkova,et al.  Integration of visual and tactile signals from the hand in the human brain: an FMRI study. , 2011, Journal of neurophysiology.

[11]  Aina Puce,et al.  Viewing the motion of human body parts activates different regions of premotor, temporal, and parietal cortex , 2004, NeuroImage.

[12]  Lora T. Likova,et al.  Stereomotion processing in the human occipital cortex , 2007, NeuroImage.

[13]  François Klam,et al.  ã Federation of European Neuroscience Societies Visual±vestibular interactive responses in the macaque ventral intraparietal area (VIP) , 2022 .

[14]  A. Doupe,et al.  Temporal and Spectral Sensitivity of Complex Auditory Neurons in the Nucleus HVc of Male Zebra Finches , 1998, The Journal of Neuroscience.

[15]  G. Orban,et al.  Common and segregated processing of observed actions in human SPL. , 2013, Cerebral cortex.

[16]  D. V. van Essen,et al.  A Population-Average, Landmark- and Surface-based (PALS) atlas of human cerebral cortex. , 2005, NeuroImage.

[17]  Robert A. Barton,et al.  Binocularity and brain evolution in primates , 2004 .

[18]  G. Orban,et al.  Psychometric curves describe action discrimination in humans. , 2015, Journal of vision.

[19]  Ravi S. Menon,et al.  Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas , 2003, Experimental Brain Research.

[20]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[21]  R. J. Seitz,et al.  A fronto‐parietal circuit for object manipulation in man: evidence from an fMRI‐study , 1999, The European journal of neuroscience.

[22]  K. Amunts,et al.  The human parietal operculum. II. Stereotaxic maps and correlation with functional imaging results. , 2006, Cerebral cortex.

[23]  Guy A. Orban,et al.  Seeing biological actions in 3D: An fMRI study , 2015, Human brain mapping.

[24]  Abigail L. Person,et al.  Pallidal Neuron Activity Increases during Sensory Relay through Thalamus in a Songbird Circuit Essential for Learning , 2007, The Journal of Neuroscience.

[25]  Sebastian M. Frank,et al.  Vestibular and visual responses in human posterior insular cortex. , 2014, Journal of neurophysiology.

[26]  G. Orban,et al.  Coding observed motor acts: different organizational principles in the parietal and premotor cortex of humans. , 2010, Journal of neurophysiology.

[27]  Philip J. Corriveau,et al.  Stereoscopic Viewing and Reported Perceived Immersion and Symptoms , 2012, Optometry and vision science : official publication of the American Academy of Optometry.

[28]  J Duysens,et al.  Neurons in the ventral intraparietal area of awake macaque monkey closely resemble neurons in the dorsal part of the medial superior temporal area in their responses to optic flow patterns. , 1996, Journal of neurophysiology.

[29]  A. Doupe,et al.  Anterior Forebrain Neurons Develop Selectivity by an Intermediate Stage of Birdsong Learning , 1997, The Journal of Neuroscience.

[30]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[31]  Mark Jenkinson,et al.  Correspondences between retinotopic areas and myelin maps in human visual cortex , 2014, NeuroImage.

[32]  A. Schleicher,et al.  Observer-independent cytoarchitectonic mapping of the human superior parietal cortex. , 2008, Cerebral cortex.

[33]  Karl J. Friston,et al.  Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution , 2003, NeuroImage.

[34]  B. P. Klein,et al.  Topographic Representation of Numerosity in the Human Parietal Cortex , 2013, Science.

[35]  Ehud Zohary,et al.  A Mirror Representation of Others' Actions in the Human Anterior Parietal Cortex , 2006, The Journal of Neuroscience.

[36]  G. Orban,et al.  A Higher Order Motion Region in Human Inferior Parietal Lobule Evidence from fMRI , 2003, Neuron.

[37]  A. Schleicher,et al.  Two different areas within the primary motor cortex of man , 1996, Nature.

[38]  Svetlana S. Georgieva,et al.  The Processing of Three-Dimensional Shape from Disparity in the Human Brain , 2009, The Journal of Neuroscience.

[39]  Jeff Miller The sampling distribution ofd′ , 1996 .

[40]  Christian Keysers,et al.  The anthropomorphic brain: The mirror neuron system responds to human and robotic actions , 2007, NeuroImage.

[41]  Hilary W. Heuer,et al.  Parietal Area VIP Neuronal Responses to Heading Stimuli Are Encoded in Head-Centered Coordinates , 2004, Neuron.

[42]  Guy A. Orban,et al.  The organization of the posterior parietal cortex devoted to upper limb actions: An fMRI study , 2015, Human brain mapping.

[43]  Vincenzo Maffei,et al.  Visual gravity cues in the interpretation of biological movements: neural correlates in humans , 2015, NeuroImage.

[44]  David C. Van Essen,et al.  A Population-Average, Landmark- and Surface-based (PALS) atlas of human cerebral cortex , 2005, NeuroImage.

[45]  Ralph Weidner,et al.  Neural Mechanisms of Attentional Reorienting in Three-Dimensional Space , 2012, The Journal of Neuroscience.

[46]  Simon B. Eickhoff,et al.  Meta-analytical definition and functional connectivity of the human vestibular cortex , 2012, NeuroImage.

[47]  S. Kastner,et al.  Two hierarchically organized neural systems for object information in human visual cortex , 2008, Nature Neuroscience.

[48]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[49]  J. Duhamel,et al.  Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey , 2007, The Journal of Neuroscience.

[50]  Ronald R. Peeters,et al.  A human homologue of monkey F5c , 2015, NeuroImage.

[51]  G. Orban The extraction of 3D shape in the visual system of human and nonhuman primates. , 2011, Annual review of neuroscience.

[52]  H. Barbas,et al.  Medial Prefrontal Cortices Are Unified by Common Connections With Superior Temporal Cortices and Distinguished by Input From Memory‐Related Areas in the Rhesus Monkey , 1999, The Journal of comparative neurology.

[53]  Doris Y. Tsao,et al.  Stereopsis Activates V3A and Caudal Intraparietal Areas in Macaques and Humans , 2003, Neuron.

[54]  Timothy Edward John Behrens,et al.  Diffusion-Weighted Imaging Tractography-Based Parcellation of the Human Lateral Premotor Cortex Identifies Dorsal and Ventral Subregions with Anatomical and Functional Specializations , 2007, The Journal of Neuroscience.

[55]  L. Fogassi,et al.  Functional properties of grasping-related neurons in the dorsal premotor area F2 of the macaque monkey. , 2004, Journal of neurophysiology.

[56]  A. Parker Binocular depth perception and the cerebral cortex , 2007, Nature Reviews Neuroscience.

[57]  J. Mazziotta,et al.  Cortical mechanisms of human imitation. , 1999, Science.

[58]  R. Lemon Descending pathways in motor control. , 2008, Annual review of neuroscience.

[59]  Marc M. Van Hulle,et al.  Optic flow from unstable sequences through local velocity constancy maximization , 2009, Image Vis. Comput..

[60]  Sabine Kastner,et al.  Functional organization of human posterior parietal cortex: grasping- and reaching-related activations relative to topographically organized cortex. , 2013, Journal of neurophysiology.

[61]  A. Solimini Are There Side Effects to Watching 3D Movies? A Prospective Crossover Observational Study on Visually Induced Motion Sickness , 2013, PloS one.

[62]  D. Regan,et al.  Hitting what one wants to hit and missing what one wants to miss , 2001, Vision Research.

[63]  R. Barton From The Cover: Binocularity and brain evolution in primates. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[64]  Scott T. Grafton,et al.  Evidence for a distributed hierarchy of action representation in the brain. , 2007, Human movement science.

[65]  Thomas Brandt,et al.  The bilateral central vestibular system: its pathways, functions, and disorders , 2015, Annals of the New York Academy of Sciences.

[66]  Arne D. Ekstrom,et al.  Single-Neuron Responses in Humans during Execution and Observation of Actions , 2010, Current Biology.

[67]  G. Rizzolatti,et al.  The Cortical Motor System , 2001, Neuron.

[68]  A. Schleicher,et al.  Cytoarchitectonic identification and probabilistic mapping of two distinct areas within the anterior ventral bank of the human intraparietal sulcus , 2006, The Journal of comparative neurology.

[69]  Y. Miyashita,et al.  Functional Magnetic Resonance Imaging of Macaque Monkeys Performing Visually Guided Saccade Tasks Comparison of Cortical Eye Fields with Humans , 2004, Neuron.

[70]  F. Lacquaniti,et al.  Representation of Visual Gravitational Motion in the Human Vestibular Cortex , 2005, Science.

[71]  Guy Orban,et al.  Processing of targets in smooth or apparent motion along the vertical in the human brain: an fMRI study. , 2010, Journal of neurophysiology.

[72]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.

[73]  Doris Y. Tsao,et al.  Single-Unit Recordings in the Macaque Face Patch System Reveal Limitations of fMRI MVPA , 2015, The Journal of Neuroscience.

[74]  Simon J Watt,et al.  The visual control of reaching and grasping: binocular disparity and motion parallax. , 2003, Journal of experimental psychology. Human perception and performance.

[75]  Guy A. Orban,et al.  Monkey Cortex through fMRI Glasses , 2014, Neuron.

[76]  Y Shinoda,et al.  Vestibular projection to the periarcuate cortex in the monkey , 2004, Neuroscience Research.

[77]  Rudolf Stark,et al.  Motor imagery of hand actions: Decoding the content of motor imagery from brain activity in frontal and parietal motor areas , 2015, Human brain mapping.

[78]  J. Kamilar,et al.  Retinogeniculostriate Pathway Components Scale with Orbit Convergence Only in Primates and Not in Other Mammals , 2011, Brain, Behavior and Evolution.

[79]  J Randall Flanagan,et al.  Planning Ahead: Object-Directed Sequential Actions Decoded from Human Frontoparietal and Occipitotemporal Networks. , 2015, Cerebral cortex.

[80]  Pierre Vanderhaeghen,et al.  Mapping Labels in the Human Developing Visual System and the Evolution of Binocular Vision , 2005, The Journal of Neuroscience.

[81]  Dylan F. Cooke,et al.  Parieto-frontal interactions, personal space, and defensive behavior , 2006, Neuropsychologia.

[82]  Richard S. J. Frackowiak,et al.  Identification of the central vestibular projections in man: a positron emission tomography activation study , 2004, Experimental Brain Research.

[83]  Bharat B. Biswal,et al.  Functional anatomy of predictive vergence and saccade eye movements in humans: A functional MRI investigation , 2010, Vision Research.

[84]  R. Turner,et al.  Characterizing Dynamic Brain Responses with fMRI: A Multivariate Approach , 1995, NeuroImage.

[85]  Katrin Amunts,et al.  The human inferior parietal cortex: Cytoarchitectonic parcellation and interindividual variability , 2006, NeuroImage.

[86]  A. Berthoz,et al.  Cortical Areas Activated by Bilateral Galvanic Vestibular Stimulation , 1999, Annals of the New York Academy of Sciences.

[87]  Bert De Smedt,et al.  Visual Number Beats Abstract Numerical Magnitude: Format-dependent Representation of Arabic Digits and Dot Patterns in Human Parietal Cortex , 2015, Journal of Cognitive Neuroscience.

[88]  O. Grüsser,et al.  Is there a vestibular cortex? , 1998, Trends in Neurosciences.

[89]  Ronald R. Peeters,et al.  Parietal regions processing visual 3D shape extracted from disparity , 2009, NeuroImage.

[90]  T. Mergner,et al.  Relationship between saccadic eye movements and cortical activity as measured by fMRI: quantitative and qualitative aspects , 2001, Experimental Brain Research.

[91]  Stefan Geyer,et al.  Prologue: Toward the Concept of a Cortical Control of Voluntary Movements , 2004 .

[92]  G. Orban,et al.  The Retinotopic Organization of the Human Middle Temporal Area MT/V5 and Its Cortical Neighbors , 2010, The Journal of Neuroscience.

[93]  G. Rizzolatti,et al.  View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex , 2011, Current Biology.

[94]  Thomas R. Knösche,et al.  Anatomical and functional parcellation of the human lateral premotor cortex , 2008, NeuroImage.

[95]  O. Grüsser,et al.  Corticofugal connections between the cerebral cortex and brainstem vestibular nuclei in the macaque monkey , 1994, The Journal of comparative neurology.

[96]  G. Orban,et al.  Motion-responsive regions of the human brain , 1999, Experimental Brain Research.

[97]  Dr. Stefan Geyer The Microstructural Border Between the Motor and the Cognitive Domain in the Human Cerebral Cortex , 2004, Advances in Anatomy Embryology and Cell Biology.

[98]  Karl J. Friston,et al.  Psychophysiological and Modulatory Interactions in Neuroimaging , 1997, NeuroImage.

[99]  Xiaomin Yue,et al.  Neural Correlates of Personal Space Intrusion , 2014, The Journal of Neuroscience.

[100]  J. Miller,et al.  The sampling distribution of d'. , 1996, Perception & psychophysics.

[101]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[102]  H. Damasio,et al.  Neural basis of action understanding: Evidence from sign language aphasia , 2013, Aphasiology.

[103]  G. Rizzolatti,et al.  Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study , 2001, The European journal of neuroscience.

[104]  Gregory C. DeAngelis,et al.  A Comparison of Vestibular Spatiotemporal Tuning in Macaque Parietoinsular Vestibular Cortex, Ventral Intraparietal Area, and Medial Superior Temporal Area , 2011, The Journal of Neuroscience.

[105]  Justine C. Cléry,et al.  Neuronal bases of peripersonal and extrapersonal spaces, their plasticity and their dynamics: Knowns and unknowns , 2015, Neuropsychologia.

[106]  Justine Cléry,et al.  Whole brain mapping of visual and tactile convergence in the macaque monkey , 2015, NeuroImage.

[107]  Philippe Kahane,et al.  Reappraisal of the human vestibular cortex by cortical electrical stimulation study , 2003, Annals of neurology.

[108]  Guy A. Orban,et al.  The overlap of the EBA and the MT/V5 cluster , 2013, NeuroImage.

[109]  G. Rizzolatti,et al.  Cortical mechanisms underlying the organization of goal-directed actions and mirror neuron-based action understanding. , 2014, Physiological reviews.

[110]  Guldin Wo,et al.  Is there a vestibular cortex , 1998 .