Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks

Brain-computer interface (BCI) technologies aim to provide a bridge between the human brain and external devices. Prior research using non-invasive BCI to control virtual objects, such as computer cursors and virtual helicopters, and real-world objects, such as wheelchairs and quadcopters, has demonstrated the promise of BCI technologies. However, controlling a robotic arm to complete reach-and-grasp tasks efficiently using non-invasive BCI has yet to be shown. In this study, we found that a group of 13 human subjects could willingly modulate brain activity to control a robotic arm with high accuracy for performing tasks requiring multiple degrees of freedom by combination of two sequential low dimensional controls. Subjects were able to effectively control reaching of the robotic arm through modulation of their brain rhythms within the span of only a few training sessions and maintained the ability to control the robotic arm over multiple months. Our results demonstrate the viability of human operation of prosthetic limbs using non-invasive BCI technology.

[1]  Yijun Wang,et al.  Visual and Auditory Brain–Computer Interfaces , 2014, IEEE Transactions on Biomedical Engineering.

[2]  José del R. Millán,et al.  Brain-Controlled Wheelchairs: A Robotic Architecture , 2013, IEEE Robotics & Automation Magazine.

[3]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[5]  N. Birbaumer,et al.  The Influence of Psychological State and Motivation on Brain–Computer Interface Performance in Patients with Amyotrophic Lateral Sclerosis – a Longitudinal Study , 2010, Front. Neuropharma..

[6]  Ning Jiang,et al.  Enhanced Low-Latency Detection of Motor Intention From EEG for Closed-Loop Brain-Computer Interface Applications , 2014, IEEE Transactions on Biomedical Engineering.

[7]  N. V. Thakor,et al.  Translating the Brain-Machine Interface , 2013, Science Translational Medicine.

[8]  John P. Cunningham,et al.  A High-Performance Neural Prosthesis Enabled by Control Algorithm Design , 2012, Nature Neuroscience.

[9]  Jie Li,et al.  A combination strategy based brain–computer interface for two-dimensional movement control , 2015, Journal of neural engineering.

[10]  Cuntai Guan,et al.  A Large Clinical Study on the Ability of Stroke Patients to Use an EEG-Based Motor Imagery Brain-Computer Interface , 2011, Clinical EEG and neuroscience.

[11]  Carmen Vidaurre,et al.  BioSig: The Free and Open Source Software Library for Biomedical Signal Processing , 2011, Comput. Intell. Neurosci..

[12]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[13]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[14]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[15]  Krishna V Shenoy,et al.  Human cortical prostheses: lost in translation? , 2009, Neurosurgical focus.

[16]  Andrés Úbeda,et al.  SVM-based Brain-Machine Interface for controlling a robot arm through four mental tasks , 2015, Neurocomputing.

[17]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Febo Cincotti,et al.  High-resolution EEG techniques for brain–computer interface applications , 2008, Journal of Neuroscience Methods.

[19]  Michel Verleysen,et al.  Feature Selection for Interpatient Supervised Heart Beat Classification , 2011, BIOSIGNALS.

[20]  J. Carmena,et al.  Emergence of a Stable Cortical Map for Neuroprosthetic Control , 2009, PLoS biology.

[21]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[22]  Takeshi Sakurada,et al.  A BMI-based occupational therapy assist suit: asynchronous control by SSVEP , 2013, Front. Neurosci..

[23]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[24]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[25]  José del R. Millán,et al.  Brain-Computer Interfaces , 2020, Handbook of Clinical Neurology.

[26]  Andrew Y. Paek,et al.  Global cortical activity predicts shape of hand during grasping , 2015, Front. Neurosci..

[27]  Christa Neuper,et al.  Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb , 2011, Medical & Biological Engineering & Computing.

[28]  G. R. Muller,et al.  Brain oscillations control hand orthosis in a tetraplegic , 2000, Neuroscience Letters.

[29]  Lei Ding,et al.  Motor imagery classification by means of source analysis for brain–computer interface applications , 2004, Journal of neural engineering.

[30]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[31]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[32]  G. Pfurtscheller,et al.  An SSVEP BCI to Control a Hand Orthosis for Persons With Tetraplegia , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[34]  Hiroshi Yokoi,et al.  Real-Time Control of a Neuroprosthetic Hand by Magnetoencephalographic Signals from Paralysed Patients , 2016, Scientific Reports.

[35]  J R Wolpaw,et al.  Spatial filter selection for EEG-based communication. , 1997, Electroencephalography and clinical neurophysiology.

[36]  Rajesh P. N. Rao,et al.  Distributed cortical adaptation during learning of a brain–computer interface task , 2013, Proceedings of the National Academy of Sciences.

[37]  G Pfurtscheller,et al.  Visualization of significant ERD/ERS patterns in multichannel EEG and ECoG data , 2002, Clinical Neurophysiology.

[38]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[39]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[40]  G. Pfurtscheller,et al.  Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. , 2005, Brain research. Cognitive brain research.