Developing a Three- to Six-State EEG-Based Brain–Computer Interface for a Virtual Robotic Manipulator Control

Objective: We develop an electroencephalography (EEG)-based noninvasive brain–computer interface (BCI) system having short training time (15 min) that can be applied for high-performance control of robotic prosthetic systems. Methods: A signal processing system for detecting user's mental intent from EEG data based on up to six-state BCI paradigm is developed and used. Results: We examine the performance of the developed system on experimental data collected from 12 healthy participants and analyzed offline. Out of 12 participants 3 achieve an accuracy of six-state communication in 80%–90% range, while 2 participants do not achieve a satisfactory accuracy. We further implement an online BCI system for control of a virtual 3 degree-of-freedom (dof) prosthetic manipulator and test it with our three best participants. Two participants are able to successfully complete 100% of the test tasks, demonstrating on average the accuracy rate of 80% and requiring 5–10 s to execute a manipulator move. One participant failed to demonstrate a satisfactory performance in online trials. Conclusion: We show that our offline EEG BCI system can correctly identify different motor imageries in EEG data with high accuracy and our online BCI system can be used for control of a virtual 3 dof prosthetic manipulator. Significance: Our results prepare foundation for further development of higher performance EEG BCI-based robotic assistive systems and demonstrate that EEG-based BCI may be feasible for robotic control by paralyzed and immobilized individuals.

[1]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[2]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[3]  Trent J. Bradberry,et al.  Fast attainment of computer cursor control with noninvasively acquired brain signals , 2011, Journal of neural engineering.

[4]  Andreas Schulze-Bonhage,et al.  Decoding natural grasp types from human ECoG , 2012, NeuroImage.

[5]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[6]  Virginia R. de Sa,et al.  Preprocessing and Meta-Classification for Brain-Computer Interfaces , 2007, IEEE Transactions on Biomedical Engineering.

[7]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[8]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[9]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[10]  E. Gysels,et al.  Phase synchronization for the recognition of mental tasks in a brain-computer interface , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[12]  D. Farina,et al.  Detection of movement intention from single-trial movement-related cortical potentials , 2011, Journal of neural engineering.

[13]  Ethan R. Buch,et al.  Think to Move: a Neuromagnetic Brain-Computer Interface (BCI) System for Chronic Stroke , 2008, Stroke.

[14]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[15]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Reinhold Scherer,et al.  A Co-Adaptive Brain-Computer Interface for End Users with Severe Motor Impairment , 2014, PloS one.

[17]  Meel Velliste,et al.  Seven Degree of Freedom Cortical Control of a Robotic Arm , 2013 .

[18]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[19]  J. Wolpaw,et al.  Decoding flexion of individual fingers using electrocorticographic signals in humans , 2009, Journal of neural engineering.

[20]  M. Teplan.  FUNDAMENTALS OF EEG MEASUREMENT , 2002 .

[21]  Bin He,et al.  Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks , 2016, Scientific Reports.

[22]  Dario Farina,et al.  Factors of Influence on the Performance of a Short-Latency Non-Invasive Brain Switch: Evidence in Healthy Individuals and Implication for Motor Function Rehabilitation , 2016, Front. Neurosci..

[23]  Klaus-Robert Müller,et al.  Classifying Single Trial EEG: Towards Brain Computer Interfacing , 2001, NIPS.

[24]  J. M. Carmena,et al.  Closed-Loop Decoder Adaptation on Intermediate Time-Scales Facilitates Rapid BMI Performance Improvements Independent of Decoder Initialization Conditions , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  José del R. Millán,et al.  Brain-Controlled Wheelchairs: A Robotic Architecture , 2013, IEEE Robotics & Automation Magazine.

[26]  C. Braun,et al.  A review on directional information in neural signals for brain-machine interfaces , 2009, Journal of Physiology-Paris.

[27]  Arjun K. Bansal,et al.  Relationships among low-frequency local field potentials, spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. , 2011, Journal of neurophysiology.

[28]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[29]  M. Thulasidas,et al.  Robust classification of EEG signal for brain-computer interface , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Sriram Subramanian,et al.  Predicting Mental Imagery-Based BCI Performance from Personality, Cognitive Profile and Neurophysiological Patterns , 2015, PloS one.

[31]  Yusuf Uzzaman Khan,et al.  Wrist movement discrimination in single-trial EEG for Brain–Computer Interface using band powers , 2011 .

[32]  Sung Chan Jun,et al.  High Theta and Low Alpha Powers May Be Indicative of BCI-Illiteracy in Motor Imagery , 2013, PloS one.

[33]  Dario Farina,et al.  Detection of movement-related cortical potentials based on subject-independent training , 2013, Medical & Biological Engineering & Computing.

[34]  Piotr Stawicki,et al.  Autonomous Parameter Adjustment for SSVEP-Based BCIs with a Novel BCI Wizard , 2015, Front. Neurosci..

[35]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[36]  José Carlos Príncipe,et al.  Coadaptive Brain–Machine Interface via Reinforcement Learning , 2009, IEEE Transactions on Biomedical Engineering.

[37]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[38]  Fusheng Yang,et al.  BCI competition 2003-data set IV:An algorithm based on CSSD and FDA for classifying single-trial EEG , 2004, IEEE Transactions on Biomedical Engineering.

[39]  Aamir Saeed Malik,et al.  Classification of Four Class Motor Imagery for Brain Computer Interface , 2017 .

[40]  Miguel A. L. Nicolelis,et al.  Brain–machine interfaces: past, present and future , 2006, Trends in Neurosciences.

[41]  G. Pfurtscheller,et al.  Information transfer rate in a five-classes brain-computer interface , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[42]  B. Kayser,et al.  Movement-Related Cortical Potential Amplitude Reduction after Cycling Exercise Relates to the Extent of Neuromuscular Fatigue , 2016, Front. Hum. Neurosci..

[43]  Ning Jiang,et al.  Detection of Movement Related Cortical Potentials from EEG Using Constrained ICA for Brain-Computer Interface Applications , 2017, Front. Neurosci..

[44]  Thomas M. Hall,et al.  A Common Structure Underlies Low-Frequency Cortical Dynamics in Movement, Sleep, and Sedation , 2014, Neuron.

[45]  C Grozea,et al.  On the feasibility of using motor imagery EEG-based brain–computer interface in chronic tetraplegics for assistive robotic arm control: a clinical test and long-term post-trial follow-up , 2012, Spinal Cord.

[46]  N. Thakor,et al.  Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand , 2010, Journal of neural engineering.

[47]  Bin He,et al.  Cortical Imaging of Event-Related (de)Synchronization During Online Control of Brain-Computer Interface Using Minimum-Norm Estimates in Frequency Domain , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[48]  K.-R. Muller,et al.  The Berlin brain-computer interface: EEG-based communication without subject training , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[49]  G Pfurtscheller,et al.  Real-time EEG analysis with subject-specific spatial patterns for a brain-computer interface (BCI). , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[50]  Klaus-Robert Müller,et al.  A regularized discriminative framework for EEG analysis with application to brain–computer interface , 2010, NeuroImage.

[51]  Nicole Krämer,et al.  Time Domain Parameters as a feature for EEG-based Brain-Computer Interfaces , 2009, Neural Networks.

[52]  José del R. Millán,et al.  Brain-Computer Interfaces , 2020, Handbook of Clinical Neurology.

[53]  Bin He,et al.  Noninvasive Brain-Computer Interfaces Based on Sensorimotor Rhythms , 2015, Proceedings of the IEEE.

[54]  Ferat Sahin,et al.  New classification techniques for electroencephalogram (EEG) signals and a real-time EEG control of a robot , 2011, Neural Computing and Applications.

[55]  Shaomin Zhang,et al.  Reliability of directional information in unsorted spikes and local field potentials recorded in human motor cortex , 2014, Journal of neural engineering.

[56]  Trent J. Bradberry,et al.  Reconstructing Three-Dimensional Hand Movements from Noninvasive Electroencephalographic Signals , 2010, The Journal of Neuroscience.

[57]  Mohsen Mollazadeh,et al.  Spatiotemporal Variation of Multiple Neurophysiological Signals in the Primary Motor Cortex during Dexterous Reach-to-Grasp Movements , 2011, The Journal of Neuroscience.

[58]  Andrew Y. Paek,et al.  Decoding repetitive finger movements with brain activity acquired via non-invasive electroencephalography , 2014, Front. Neuroeng..

[59]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[60]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.