Multivariate Analysis of Electrophysiological Signals Reveals the Temporal Properties of Visuomotor Computations for Precision Grips

The frontoparietal networks underlying grasping movements have been extensively studied, especially using fMRI. Accordingly, whereas much is known about their cortical locus much less is known about the temporal dynamics of visuomotor transformations. Here, we show that multivariate EEG analysis allows for detailed insights into the time course of visual and visuomotor computations of precision grasps. The frontoparietal networks underlying grasping movements have been extensively studied, especially using fMRI. Accordingly, whereas much is known about their cortical locus much less is known about the temporal dynamics of visuomotor transformations. Here, we show that multivariate EEG analysis allows for detailed insights into the time course of visual and visuomotor computations of precision grasps. Male and female human participants first previewed one of several objects and, upon its reappearance, reached to grasp it with the thumb and index finger along one of its two symmetry axes. Object shape classifiers reached transient accuracies of 70% at ∼105 ms, especially based on scalp sites over visual cortex, dropping to lower levels thereafter. Grasp orientation classifiers relied on a system of occipital-to-frontal electrodes. Their accuracy rose concurrently with shape classification but ramped up more gradually, and the slope of the classification curve predicted individual reaction times. Further, cross-temporal generalization revealed that dynamic shape representation involved early and late neural generators that reactivated one another. In contrast, grasp computations involved a chain of generators attaining a sustained state about 100 ms before movement onset. Our results reveal the progression of visual and visuomotor representations over the course of planning and executing grasp movements. SIGNIFICANCE STATEMENT Grasping an object requires the brain to perform visual-to-motor transformations of the object's properties. Although much of the neuroanatomic basis of visuomotor transformations has been uncovered, little is known about its time course. Here, we orthogonally manipulated object visual characteristics and grasp orientation, and used multivariate EEG analysis to reveal that visual and visuomotor computations follow similar time courses but display different properties and dynamics.

[1]  Luca Turella,et al.  MEG Multivariate Analysis Reveals Early Abstract Action Representations in the Lateral Occipitotemporal Cortex , 2015, The Journal of Neuroscience.

[2]  U. Castiello The neuroscience of grasping , 2005, Nature Reviews Neuroscience.

[3]  M. Illert,et al.  Development of prehension movements in children: a kinematic study , 1998, Experimental Brain Research.

[4]  J. Pereira,et al.  Decoding natural reach-and-grasp actions from human EEG , 2018, Journal of neural engineering.

[5]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[6]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[7]  Dorothy V. M. Bishop,et al.  Journal of Neuroscience Methods , 2015 .

[8]  Melvyn A. Goodale,et al.  The dissociation between perception and action in the Ebbinghaus illusion Nonillusory effects of pictorial cues on grasp , 2001, Current Biology.

[9]  Hansjörg Scherberger,et al.  Population coding of grasp and laterality-related information in the macaque fronto-parietal network , 2017, Scientific Reports.

[10]  Teresa D Figley,et al.  Recruitment of Foveal Retinotopic Cortex During Haptic Exploration of Shapes and Actions in the Dark , 2017, The Journal of Neuroscience.

[11]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[12]  Hansjörg Scherberger,et al.  Context-Specific Grasp Movement Representation in the Macaque Anterior Intraparietal Area , 2009, The Journal of Neuroscience.

[13]  Rhodri Cusack,et al.  Disentangling Representations of Object and Grasp Properties in the Human Brain , 2016, The Journal of Neuroscience.

[14]  P. Downing,et al.  The lateral occipitotemporal cortex in action , 2015, Trends in Cognitive Sciences.

[15]  Scott T Grafton,et al.  Differential Recruitment of Anterior Intraparietal Sulcus and Superior Parietal Lobule during Visually Guided Grasping Revealed by Electrical Neuroimaging , 2008, The Journal of Neuroscience.

[16]  Ravi S. Menon,et al.  Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas , 2003, Experimental Brain Research.

[17]  K. Gegenfurtner,et al.  Differences in fixations between grasping and viewing objects. , 2009, Journal of vision.

[18]  Jody C. Culham,et al.  Ventral and dorsal stream contributions to the online control of immediate and delayed grasping: A TMS approach , 2009, Neuropsychologia.

[19]  J Randall Flanagan,et al.  Where One Hand Meets the Other: Limb-Specific and Action-Dependent Movement Plans Decoded from Preparatory Signals in Single Human Frontoparietal Brain Areas , 2013, The Journal of Neuroscience.

[20]  Hein T. van Schie,et al.  Neural mechanisms underlying immediate and final action goals in object use reflected by slow wave brain potentials , 2007, Brain Research.

[21]  R D Pascual-Marqui,et al.  Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. , 2002, Methods and findings in experimental and clinical pharmacology.

[22]  S. Huettel,et al.  A nexus model of the temporal–parietal junction , 2013, Trends in Cognitive Sciences.

[23]  Bruno B Averbeck,et al.  Rapid Sequences of Population Activity Patterns Dynamically Encode Task-Critical Spatial Information in Parietal Cortex , 2010, The Journal of Neuroscience.

[24]  Matthias Niemeier,et al.  The time course of individual face recognition: A pattern analysis of ERP signals , 2016, NeuroImage.

[25]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[26]  Jody C Culham,et al.  Functional magnetic resonance adaptation reveals the involvement of the dorsomedial stream in hand orientation for grasping. , 2011, Journal of neurophysiology.

[27]  Hansjörg Scherberger,et al.  Object vision to hand action in macaque parietal, premotor, and motor cortices , 2016, eLife.

[28]  C. Galletti,et al.  Evidence for both reaching and grasping activity in the medial parieto‐occipital cortex of the macaque , 2004, The European journal of neuroscience.

[29]  Steven J. Luck,et al.  ERPLAB: an open-source toolbox for the analysis of event-related potentials , 2014, Front. Hum. Neurosci..

[30]  J. Culham,et al.  Neural coding within human brain areas involved in actions , 2015, Current Opinion in Neurobiology.

[31]  Peter Janssen,et al.  Selectivity for three-dimensional contours and surfaces in the anterior intraparietal area. , 2012, Journal of neurophysiology.

[32]  Jody C. Culham,et al.  Human neuroimaging reveals the subcomponents of grasping, reaching and pointing actions , 2018, Cortex.

[33]  Kenneth F. Valyear,et al.  Decoding Action Intentions from Preparatory Brain Activity in Human Parieto-Frontal Networks , 2011, The Journal of Neuroscience.

[34]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[35]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[36]  Radoslaw Martin Cichy,et al.  Resolving human object recognition in space and time , 2014, Nature Neuroscience.

[37]  Arthur W. Toga,et al.  A Probabilistic Atlas of the Human Brain: Theory and Rationale for Its Development The International Consortium for Brain Mapping (ICBM) , 1995, NeuroImage.

[38]  J. Culham,et al.  The human dorsal stream adapts to real actions and 3D shape processing: a functional magnetic resonance imaging study. , 2008, Journal of neurophysiology.

[39]  Marcel A. J. van Gerven,et al.  Eye movements explain decodability during perception and cued attention in MEG , 2019, NeuroImage.

[40]  Fraser W. Smith,et al.  Decoding Effector-Dependent and Effector-Independent Movement Intentions from Human Parieto-Frontal Brain Activity , 2011, The Journal of Neuroscience.

[41]  Patrizia Fattori,et al.  Hand Orientation during Reach-to-Grasp Movements Modulates Neuronal Activity in the Medial Posterior Parietal Area V6A , 2009, The Journal of Neuroscience.

[42]  Michael A. Arbib,et al.  Modeling parietal-premotor interactions in primate control of grasping , 1998, Neural Networks.

[43]  M. Davare,et al.  Temporal Dissociation between Hand Shaping and Grip Force Scaling in the Anterior Intraparietal Area , 2007, The Journal of Neuroscience.

[44]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[45]  Brandi Lee Drisdelle,et al.  Dealing with ocular artifacts on lateralized ERPs in studies of visual-spatial attention and memory: ICA correction versus epoch rejection. , 2017, Psychophysiology.

[46]  Paul Boon,et al.  EEG source localization of the epileptogenic focus in patients with refractory temporal lobe epilepsy, dipole modelling revisited. , 2007, Acta neurologica Belgica.

[47]  M. Corbetta,et al.  Extrastriate body area in human occipital cortex responds to the performance of motor actions , 2004, Nature Neuroscience.

[48]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[49]  Á. Pascual-Leone,et al.  Linking Out-of-Body Experience and Self Processing to Mental Own-Body Imagery at the Temporoparietal Junction , 2005, The Journal of Neuroscience.

[50]  Matthias Niemeier,et al.  Multimodal evidence on shape and surface information in individual face processing , 2018 .

[51]  H. Sakata,et al.  Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. , 2000, Journal of neurophysiology.

[52]  Matthias Niemeier,et al.  The right anterior intraparietal sulcus is critical for bimanual grasping: a TMS study. , 2014, Cerebral cortex.

[53]  H. Sakata,et al.  Parietal neurons related to memory-guided hand manipulation. , 1996, Journal of neurophysiology.

[54]  Dirk Koester,et al.  Neurophysiology of Grasping Actions: Evidence from ERPs , 2016, Front. Psychol..

[55]  Angelika Lingnau,et al.  Time-resolved decoding of planned delayed and immediate prehension movements , 2018, Cortex.

[56]  Susan G. Wardle,et al.  Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data , 2016, Journal of Cognitive Neuroscience.

[57]  Mads Jochumsen,et al.  Detecting and classifying three different hand movement types through electroencephalography recordings for neurorehabilitation , 2015, Medical & Biological Engineering & Computing.

[58]  Umberto Castiello,et al.  Co-Registering Kinematics and Evoked Related Potentials during Visually Guided Reach-to-Grasp Movements , 2013, PloS one.

[59]  M. Levin,et al.  The development of coordination for reach-to-grasp movements in children , 2002, Experimental Brain Research.

[60]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[61]  Scott T. Grafton,et al.  Virtual lesions of the anterior intraparietal area disrupt goal-dependent on-line adjustments of grasp , 2005, Nature Neuroscience.

[62]  M. Arbib,et al.  Grasping objects: the cortical mechanisms of visuomotor transformation , 1995, Trends in Neurosciences.

[63]  Naznin Virji-Babul,et al.  Spatial‐temporal dynamics of cortical activity underlying reaching and grasping , 2009, Human brain mapping.

[64]  Ricardo Chavarriaga,et al.  Human EEG reveals distinct neural correlates of power and precision grasping types , 2018, NeuroImage.

[65]  Jonathan S. Cant,et al.  Scratching Beneath the Surface: New Insights into the Functional Properties of the Lateral Occipital Area and Parahippocampal Place Area , 2011, The Journal of Neuroscience.

[66]  David J. Freedman,et al.  Dynamic population coding of category information in inferior temporal and prefrontal cortex. , 2008, Journal of neurophysiology.

[67]  J L Contreras-Vidal,et al.  Multisession, noninvasive closed-loop neuroprosthetic control of grasping by upper limb amputees. , 2016, Progress in brain research.

[68]  Julius P. A. Dewald,et al.  Evaluation of different cortical source localization methods using simulated and experimental EEG data , 2005, NeuroImage.

[69]  Philippe A. Chouinard,et al.  Sensitivity to biomechanical limitations during postural decision-making depends on the integrity of posterior superior parietal cortex , 2017, Cortex.

[70]  Katja Fiehler,et al.  Working memory maintenance of grasp-target information in the human posterior parietal cortex , 2011, NeuroImage.

[71]  J. Fermaglich Electric Fields of the Brain: The Neurophysics of EEG , 1982 .

[72]  Afra Wohlschläger,et al.  The Neural Correlates of Planning and Executing Actual Tool Use , 2014, The Journal of Neuroscience.

[73]  Andrew Y. Paek,et al.  Global cortical activity predicts shape of hand during grasping , 2015, Front. Neurosci..

[74]  Thomas Brochier,et al.  Planning of visually guided reach-to-grasp movements: inference from reaction time and contingent negative variation (CNV). , 2012, Psychophysiology.

[75]  Liam D. Kaufman,et al.  Human fMRI Reveals That Delayed Action Re-Recruits Visual Perception , 2013, PloS one.

[76]  M. Goodale,et al.  Visual control of action but not perception requires analytical processing of object shape , 2003, Nature.

[77]  Luca Turella,et al.  Beta band modulations underlie action representations for movement planning , 2016, NeuroImage.

[78]  Anke Marit Albers,et al.  Eye Movement-Related Confounds in Neural Decoding of Visual Working Memory Representations , 2017, eNeuro.

[79]  Ivan Toni,et al.  Cortical Dynamics of Sensorimotor Integration during Grasp Planning , 2012, The Journal of Neuroscience.

[80]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[81]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[82]  Matthias Niemeier,et al.  The Neural Dynamics of Facial Identity Processing: Insights from EEG-Based Pattern Analysis and Image Reconstruction , 2018, eNeuro.

[83]  Dirk Koester,et al.  Event-Related Brain Potentials for Goal-Related Power Grips , 2013, PloS one.

[84]  M. Goodale,et al.  Perceptual illusion and the real-time control of action. , 2003, Spatial vision.

[85]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[86]  Daniel J. Mitchell,et al.  The time-course of component processes of selective attention , 2019, NeuroImage.

[87]  Jody C Culham,et al.  Decoding the neural mechanisms of human tool use , 2013, eLife.

[88]  Patrizia Fattori,et al.  The Dorsomedial Pathway Is Not Just for Reaching: Grasping Neurons in the Medial Parieto-Occipital Cortex of the Macaque Monkey , 2010, The Journal of Neuroscience.

[89]  D Lehmann,et al.  Source localization of EEG activity during hypnotically induced anxiety and relaxation. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.