Decoding natural grasp types from human ECoG

Electrocorticographic (ECoG) signals have been successfully used to provide information about arm movement direction, individual finger movements and even continuous arm movement trajectories. Thus, ECoG has been proposed as a potential control signal for implantable brain-machine interfaces (BMIs) in paralyzed patients. For the neuronal control of a prosthesis with versatile hand/arm functions, it is also necessary to successfully decode different types of grasping movements, such as precision grip and whole-hand grip. Although grasping is one of the most frequent and important hand movements performed in everyday life, until now, the decoding of ECoG activity related to different grasp types has not been systematically investigated. Here, we show that two different grasp types (precision vs. whole-hand grip) can be reliably distinguished in natural reach-to-grasp movements in single-trial ECoG recordings from the human motor cortex. Self-paced movement execution in a paradigm accounting for variability in grasped object position and weight was chosen to create a situation similar to everyday settings. We identified three informative signal components (low-pass-filtered component, low-frequency and high-frequency amplitude modulations), which allowed for accurate decoding of precision and whole-hand grips. Importantly, grasp type decoding generalized over different object positions and weights. Within the frontal lobe, informative signals predominated in the precentral motor cortex and could also be found in the right hemisphere's homologue of Broca's area. We conclude that ECoG signals are promising candidates for BMIs that include the restoration of grasping movements.

[1]  C. Mehring,et al.  Comparing information about arm movement direction in single channels of local and epicortical field potentials from monkey and human motor cortex , 2004, Journal of Physiology-Paris.

[2]  Tonio Ball,et al.  Towards an implantable brain-machine interface based on epicortical field potentials , 2004 .

[3]  C. Braun,et al.  A review on directional information in neural signals for brain-machine interfaces , 2009, Journal of Physiology-Paris.

[4]  K. Zilles,et al.  Neural activity in human primary motor cortex areas 4a and 4p is modulated differentially by attention to action. , 2002, Journal of neurophysiology.

[5]  Philippe Kahane,et al.  High gamma frequency oscillatory activity dissociates attention from intention in the human premotor cortex , 2005, NeuroImage.

[6]  Hansjörg Scherberger,et al.  Context-Specific Grasp Movement Representation in the Macaque Anterior Intraparietal Area , 2009, The Journal of Neuroscience.

[7]  Amy J Bastian,et al.  Multidigit Movement Synergies of the Human Hand in an Unconstrained Haptic Exploration Task , 2008, The Journal of Neuroscience.

[8]  Sven Bestmann,et al.  New approaches to the study of human brain networks underlying spatial attention and related processes , 2010, Experimental Brain Research.

[9]  M. Davare,et al.  Temporal Dissociation between Hand Shaping and Grip Force Scaling in the Anterior Intraparietal Area , 2007, The Journal of Neuroscience.

[10]  E. Fetz,et al.  Oscillatory activity in sensorimotor cortex of awake monkeys: synchronization of local field potentials and relation to behavior. , 1996, Journal of neurophysiology.

[11]  D. Kovalev,et al.  Rapid and fully automated visualization of subdural electrodes in the presurgical evaluation of epilepsy patients. , 2005, AJNR. American journal of neuroradiology.

[12]  S P Levine,et al.  A direct brain interface based on event-related potentials. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[13]  J. F. Soechting,et al.  Force synergies for multifingered grasping , 2000, Experimental Brain Research.

[14]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[15]  C. Braun,et al.  Hand Movement Direction Decoded from MEG and EEG , 2008, The Journal of Neuroscience.

[16]  E. Fetz,et al.  Decoupling the Cortical Power Spectrum Reveals Real-Time Representation of Individual Finger Movements in Humans , 2009, The Journal of Neuroscience.

[17]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[18]  Philippe Kahane,et al.  Task‐related gamma‐band dynamics from an intracerebral perspective: Review and implications for surface EEG and MEG , 2009, Human brain mapping.

[19]  O Levrier,et al.  Location of hand function in the sensorimotor cortex: MR and functional correlation. , 1994, AJNR. American journal of neuroradiology.

[20]  Klaus-Robert Müller,et al.  Machine learning for real-time single-trial EEG-analysis: From brain–computer interfacing to mental state monitoring , 2008, Journal of Neuroscience Methods.

[21]  Colum D. MacKinnon,et al.  The effect of temporal accuracy constraints on movement-related potentials , 2009, Experimental Brain Research.

[22]  N. Crone,et al.  Cortical γ responses: searching high and low. , 2011, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[23]  C. Mehring,et al.  Encoding of Movement Direction in Different Frequency Ranges of Motor Cortical Local Field Potentials , 2005, The Journal of Neuroscience.

[24]  M. Latash,et al.  Enslaving effects in multi-finger force production , 2000, Experimental Brain Research.

[25]  Dimitrios Pantazis,et al.  Coherent neural representation of hand speed in humans revealed by MEG imaging , 2007, Proceedings of the National Academy of Sciences.

[26]  C. Mehring,et al.  Differential Representation of Arm Movement Direction in Relation to Cortical Anatomy and Function , 2008 .

[27]  J B Poline,et al.  Partially overlapping neural networks for real and imagined hand movements. , 2000, Cerebral cortex.

[28]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[29]  José Luis Contreras-Vidal,et al.  Decoding center-out hand velocity from MEG signals during visuomotor adaptation , 2009, NeuroImage.

[30]  Hideki Shimazu,et al.  Modulation of primary motor cortex outputs from ventral premotor cortex during visually guided grasp in the macaque monkey , 2009, The Journal of physiology.

[31]  D. Thomson,et al.  Spectrum estimation and harmonic analysis , 1982, Proceedings of the IEEE.

[32]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[33]  C. Mehring,et al.  Inference of hand movements from local field potentials in monkey motor cortex , 2003, Nature Neuroscience.

[34]  E. Fetz,et al.  Direct control of paralyzed muscles by cortical neurons , 2008, Nature.

[35]  T. Mergner,et al.  Selecting relevant electrode positions for classification tasks based on the electro-encephalogram , 2006, Medical and Biological Engineering and Computing.

[36]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[37]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[38]  H. Steinmetz,et al.  Craniocerebral topography within the international 10-20 system. , 1989, Electroencephalography and clinical neurophysiology.

[39]  Naotaka Fujii,et al.  Long-Term Asynchronous Decoding of Arm Motion Using Electrocorticographic Signals in Monkeys , 2009, Front. Neuroeng..

[40]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[41]  Simon B. Eickhoff,et al.  Testing anatomically specified hypotheses in functional imaging using cytoarchitectonic maps , 2006, NeuroImage.

[42]  Wei Wu,et al.  Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter , 2006, Neural Computation.

[43]  M. Rushworth,et al.  The left parietal and premotor cortices: motor attention and selection , 2003, NeuroImage.

[44]  Mohsen Mollazadeh,et al.  Spectral modulation of LFP activity in M1 during dexterous finger movements , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[45]  P H Peckham,et al.  EEG-based control of a hand grasp neuroprosthesis. , 1999, Neuroreport.

[46]  Heidi Johansen-Berg,et al.  Attention to movement modulates activity in sensori-motor areas, including primary motor cortex , 2001, Experimental Brain Research.

[47]  J. Wolpaw,et al.  Decoding flexion of individual fingers using electrocorticographic signals in humans , 2009, Journal of neural engineering.

[48]  Donald B. Percival,et al.  Spectral Analysis for Physical Applications: Harmonic Analysis , 1993 .

[49]  K. Zilles,et al.  Illusory Arm Movements Activate Cortical Motor Areas: A Positron Emission Tomography Study , 1999, The Journal of Neuroscience.

[50]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[51]  Alexander Kraskov,et al.  Selectivity for Grasp in Local Field Potential and Single Neuron Activity Recorded Simultaneously from M1 and F5 in the Awake Macaque Monkey , 2008, The Journal of Neuroscience.

[52]  Trent J. Bradberry,et al.  Reconstructing Three-Dimensional Hand Movements from Noninvasive Electroencephalographic Signals , 2010, The Journal of Neuroscience.

[53]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[54]  Bernhard Schölkopf,et al.  Support vector channel selection in BCI , 2004, IEEE Transactions on Biomedical Engineering.

[55]  J. Wolpaw,et al.  Decoding two-dimensional movement trajectories using electrocorticographic signals in humans , 2007, Journal of neural engineering.

[56]  Andreas Schulze-Bonhage,et al.  Prediction of arm movement trajectories from ECoG-recordings in humans , 2008, Journal of Neuroscience Methods.

[57]  José Carlos Príncipe,et al.  Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans: Motor control features , 2009, Neural Networks.

[58]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[59]  Eran Stark,et al.  Predicting Movement from Multiunit Activity , 2007, The Journal of Neuroscience.

[60]  Christa Neuper,et al.  Motor imagery and EEG-based control of spelling devices and neuroprostheses. , 2006, Progress in brain research.

[61]  J. Kaiser,et al.  Human gamma-frequency oscillations associated with attention and memory , 2007, Trends in Neurosciences.

[62]  Donald B. Percival,et al.  Spectral Analysis for Physical Applications , 1993 .

[63]  R. Andersen,et al.  Cortical Local Field Potential Encodes Movement Intentions in the Posterior Parietal Cortex , 2005, Neuron.

[64]  G. R. Muller,et al.  Brain oscillations control hand orthosis in a tetraplegic , 2000, Neuroscience Letters.

[65]  S. Kastner,et al.  Complex organization of human primary motor cortex: a high-resolution fMRI study. , 2008, Journal of neurophysiology.

[66]  Justin L. Gardner,et al.  Executed and Observed Movements Have Different Distributed Representations in Human aIPS , 2008, The Journal of Neuroscience.

[67]  A P Georgopoulos,et al.  Effects of optic flow in motor cortex and area 7a. , 2001, Journal of neurophysiology.

[68]  Leonardo Fogassi,et al.  Anterior intraparietal cortex codes complexity of observed hand movements , 2010, Brain Research Bulletin.

[69]  J. Friedman Regularized Discriminant Analysis , 1989 .

[70]  Andreas Schulze-Bonhage,et al.  Signal quality of simultaneously recorded invasive and non-invasive EEG , 2009, NeuroImage.

[71]  H. Alkadhi,et al.  Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. , 1997, Brain : a journal of neurology.

[72]  M. Schieber,et al.  How somatotopic is the motor cortex hand area? , 1993, Science.

[73]  Rajesh P. N. Rao,et al.  Electrocorticography-based brain computer Interface-the seattle experience , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[74]  Andreas Schulze-Bonhage,et al.  sLORETA allows reliable distributed source reconstruction based on subdural strip and grid recordings , 2012, Human brain mapping.

[75]  Carolyn R. Mason,et al.  Hand synergies during reach-to-grasp. , 2001, Journal of neurophysiology.

[76]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[77]  Eric Leuthardt,et al.  Real-time detection of event-related brain activity , 2008, NeuroImage.