Decoding natural reach-and-grasp actions from human EEG

OBJECTIVE Despite the high number of degrees of freedom of the human hand, most actions of daily life can be executed incorporating only palmar, pincer and lateral grasp. In this study we attempt to discriminate these three different executed reach-and-grasp actions utilizing their EEG neural correlates. APPROACH In a cue-guided experiment, 15 healthy individuals were asked to perform these actions using daily life objects. We recorded 72 trials for each reach-and-grasp condition and from a no-movement condition. MAIN RESULTS Using low-frequency time domain features from 0.3 to 3 Hz, we achieved binary classification accuracies of 72.4%, STD  ±  5.8% between grasp types, for grasps versus no-movement condition peak performances of 93.5%, STD  ±  4.6% could be reached. In an offline multiclass classification scenario which incorporated not only all reach-and-grasp actions but also the no-movement condition, the highest performance could be reached using a window of 1000 ms for feature extraction. Classification performance peaked at 65.9%, STD  ±  8.1%. Underlying neural correlates of the reach-and-grasp actions, investigated over the primary motor cortex, showed significant differences starting from approximately 800 ms to 1200 ms after the movement onset which is also the same time frame where classification performance reached its maximum. SIGNIFICANCE We could show that it is possible to discriminate three executed reach-and-grasp actions prominent in people's everyday use from non-invasive EEG. Underlying neural correlates showed significant differences between all tested conditions. These findings will eventually contribute to our attempt of controlling a neuroprosthesis in a natural and intuitive way, which could ultimately benefit motor impaired end users in their daily life actions.

[1]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[2]  Accepted April,et al.  Factors affecting higher-order movement planning: a kinematic analysis of human prehension , 1991 .

[3]  G Pfurtscheller,et al.  EEG-based communication: improved accuracy by response verification. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[4]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[5]  Michael G. Lacourse,et al.  Cortical potentials during imagined movements in individuals with chronic spinal cord injuries , 1999, Behavioural Brain Research.

[6]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[7]  K. Kilgore,et al.  An advanced neuroprosthesis for restoration of hand and upper arm control using an implantable controller. , 2002, The Journal of hand surgery.

[8]  G. Pfurtscheller,et al.  ‘Thought’ – control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia , 2003, Neuroscience Letters.

[9]  Karl M. Newell,et al.  Perceived effort in force production as reflected in motor-related cortical potentials , 2004, Clinical Neurophysiology.

[10]  Maarten J. IJzerman,et al.  Survey of the needs of patients with spinal cord injury: impact and priority for improvement in hand function in tetraplegics , 2004, Spinal Cord.

[11]  M. A. Goodale,et al.  Factors affecting higher-order movement planning: a kinematic analysis of human prehension , 2004, Experimental Brain Research.

[12]  K. Anderson Targeting recovery: priorities of the spinal cord-injured population. , 2004, Journal of neurotrauma.

[13]  Toshio Moritani,et al.  Movement-related cortical potentials during handgrip contractions with special reference to force and electromyogram bilateral deficit , 2004, European Journal of Applied Physiology and Occupational Physiology.

[14]  G. Pfurtscheller,et al.  EEG-based neuroprosthesis control: A step towards clinical practice , 2005, Neuroscience Letters.

[15]  U. Castiello The neuroscience of grasping , 2005, Nature Reviews Neuroscience.

[16]  M. Hallett,et al.  What is the Bereitschaftspotential? , 2006, Clinical Neurophysiology.

[17]  Gernot R. Müller-Putz,et al.  Brain-computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation / Brain-Computer Interfaces zur Steuerung von Neuroprothesen: von der synchronen zur asynchronen Funktionsweise , 2006 .

[18]  G. Pfurtscheller,et al.  Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  G. Pfurtscheller,et al.  Brain-computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation. , 2006, Biomedizinische Technik. Biomedical engineering.

[20]  R. Ward,et al.  EMG and EOG artifacts in brain computer interface systems: A survey , 2007, Clinical Neurophysiology.

[21]  R Rupp,et al.  Neuroprosthetics of the upper extremity--clinical application in spinal cord injury and challenges for the future. , 2007, Acta neurochirurgica. Supplement.

[22]  G. Pfurtscheller,et al.  A fully automated correction method of EOG artifacts in EEG recordings , 2007, Clinical Neurophysiology.

[23]  Francisco Sepulveda,et al.  Delta band contribution in cue based single trial classification of real and imaginary wrist movements , 2008, Medical & Biological Engineering & Computing.

[24]  Clemens Brunner,et al.  Better than random? A closer look on BCI results , 2008 .

[25]  T. Sinkjær,et al.  Central nervous system lesions leading to disability , 2008 .

[26]  C. Braun,et al.  A review on directional information in neural signals for brain-machine interfaces , 2009, Journal of Physiology-Paris.

[27]  Kim D Anderson,et al.  Consideration of user priorities when developing neural prosthetics , 2009, Journal of neural engineering.

[28]  Marie-Françoise Lucas,et al.  Identification of task parameters from movement-related cortical potentials , 2009, Medical & Biological Engineering & Computing.

[29]  Dario Farina,et al.  Single-trial discrimination of type and speed of wrist movements from EEG recordings , 2009, Clinical Neurophysiology.

[30]  Christa Neuper,et al.  Temporal Coding of Brain Patterns for Direct Limb Control in Humans , 2010, Front. Neurosci..

[31]  D. Farina,et al.  Detection of movement intention from single-trial movement-related cortical potentials , 2011, Journal of neural engineering.

[32]  A. Kübler,et al.  Flashing characters with famous faces improves ERP-based brain–computer interface performance , 2011, Journal of neural engineering.

[33]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[34]  Andreas Schulze-Bonhage,et al.  Decoding natural grasp types from human ECoG , 2012, NeuroImage.

[35]  Jason Farquhar,et al.  Detection of event-related desynchronization during attempted and imagined movements in tetraplegics for brain switch control , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[36]  Christa Neuper,et al.  Proposing a Standardized Protocol for Raw Biosignal Transmission , 2012, IEEE Transactions on Biomedical Engineering.

[37]  Christa Neuper,et al.  Autocalibration and Recurrent Adaptation: Towards a Plug and Play Online ERD-BCI , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[38]  Brendan Z. Allison,et al.  Is It Significant? Guidelines for Reporting BCI Performance , 2012 .

[39]  Maureen Clerc,et al.  An analysis of performance evaluation for motor-imagery based BCI , 2013, Journal of neural engineering.

[40]  Zhaohui Wu,et al.  The Convergence of Machine and Biological Intelligence , 2013, IEEE Intelligent Systems.

[41]  C. Neuper,et al.  Whatever Works: A Systematic User-Centered Training Protocol to Optimize Brain-Computer Interfacing Individually , 2013, PloS one.

[42]  Vera Kaiser,et al.  Hybrid brain-computer interfaces and hybrid neuroprostheses for restoration of upper limb functions in individuals with high-level spinal cord injury , 2013, Artif. Intell. Medicine.

[43]  Andreas Schulze-Bonhage,et al.  Grasp Detection from Human ECoG during Natural Reach-to-Grasp Movements , 2013, PloS one.

[44]  Mads Jochumsen,et al.  Detection and classification of movement-related cortical potentials associated with task force and speed , 2013, Journal of neural engineering.

[45]  Ning Jiang,et al.  Enhanced Low-Latency Detection of Motor Intention From EEG for Closed-Loop Brain-Computer Interface Applications , 2014, IEEE Transactions on Biomedical Engineering.

[46]  Gernot R. Müller-Putz,et al.  Brain-controlled applications using dynamic P300 speller matrices , 2015, Artif. Intell. Medicine.

[47]  Gernot R. Müller-Putz,et al.  Functional Rehabilitation of the Paralyzed Upper Extremity After Spinal Cord Injury by Noninvasive Hybrid Neuroprostheses , 2015, Proceedings of the IEEE.

[48]  Gernot R. Müller-Putz,et al.  Game-Based BCI Training: Interactive Design for Individuals with Cerebral Palsy , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[49]  J. Farquhar,et al.  Lost in time... The search for intentions and Readiness Potentials , 2015, Consciousness and Cognition.

[50]  Josef Faller,et al.  Control or non-control state: that is the question! An asynchronous visual P300-based BCI approach , 2015, Journal of neural engineering.

[51]  Dario Farina,et al.  Detecting and classifying movement-related cortical potentials associated with hand movements in healthy subjects and stroke patients from single-electrode, single-trial EEG , 2015, Journal of neural engineering.

[52]  Mads Jochumsen,et al.  Detecting and classifying three different hand movement types through electroencephalography recordings for neurorehabilitation , 2015, Medical & Biological Engineering & Computing.

[53]  Andrew Y. Paek,et al.  Global cortical activity predicts shape of hand during grasping , 2015, Front. Neurosci..

[54]  Jason Farquhar,et al.  Detection of attempted movement from the EEG during neuromuscular block: proof of principle study in awake volunteers , 2015, Scientific Reports.

[55]  Ricardo Chavarriaga,et al.  Detecting intention to grasp during reaching movements from EEG , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[56]  Gernot R. Müller-Putz,et al.  A co-adaptive sensory motor rhythms Brain-Computer Interface based on common spatial patterns and Random Forest , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[57]  Mads Jochumsen,et al.  Comparison of spatial filters and features for the detection and classification of movement-related cortical potentials in healthy individuals and stroke patients , 2015, Journal of neural engineering.

[58]  Reinhold Scherer,et al.  Thought-based row-column scanning communication board for individuals with cerebral palsy. , 2015, Annals of physical and rehabilitation medicine.

[59]  Gernot R. Müller-Putz,et al.  Lets play Tic-Tac-Toe: A Brain-Computer Interface case study in cerebral palsy , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[60]  Andreas Schwarz Time domain classification of grasp and hold tasks , 2016 .

[61]  Gernot R. Müller-Putz,et al.  Brain-computer interface adaptation for an end user to compete in the Cybathlon , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[62]  J L Contreras-Vidal,et al.  Multisession, noninvasive closed-loop neuroprosthetic control of grasping by upper limb amputees. , 2016, Progress in brain research.

[63]  Gernot R. Müller-Putz,et al.  Movements of the same upper limb can be classified from low-frequency time-domain EEG signals , 2016 .

[64]  Ricardo Chavarriaga,et al.  Decoding of two hand grasping types from EEG , 2016 .

[65]  Gernot R. Müller-Putz,et al.  Single Versus Multiple Events Error Potential Detection in a BCI-Controlled Car Game With Continuous and Discrete Feedback , 2016, IEEE Transactions on Biomedical Engineering.

[66]  Gernot R. Müller-Putz,et al.  Hierarchical decoding of grasping commands from EEG , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[67]  Luis Montesano,et al.  Moregrasp: restoration of upper limb function in Individuals with High spinal cord injury by Multimodal Neuroprostheses for Interaction in Daily Activities , 2017, GBCIC.