Evaluation of a P300-Based Brain-Machine Interface for a Robotic Hand-Orthosis Control

This work presents the design, implementation, and evaluation of a P300-based brain-machine interface (BMI) developed to control a robotic hand-orthosis. The purpose of this system is to assist patients with amyotrophic lateral sclerosis (ALS) who cannot open and close their hands by themselves. The user of this interface can select one of six targets, which represent the flexion-extension of one finger independently or the movement of the five fingers simultaneously. We tested offline and online our BMI on eighteen healthy subjects (HS) and eight ALS patients. In the offline test, we used the calibration data of each participant recorded in the experimental sessions to estimate the accuracy of the BMI to classify correctly single epochs as target or non-target trials. On average, the system accuracy was 78.7% for target epochs and 85.7% for non-target trials. Additionally, we observed significant P300 responses in the calibration recordings of all the participants, including the ALS patients. For the BMI online test, each subject performed from 6 to 36 attempts of target selections using the interface. In this case, around 46% of the participants obtained 100% of accuracy, and the average online accuracy was 89.83%. The maximum information transfer rate (ITR) observed in the experiments was 52.83 bit/min, whereas that the average ITR was 18.13 bit/min. The contributions of this work are the following. First, we report the development and evaluation of a mind-controlled robotic hand-orthosis for patients with ALS. To our knowledge, this BMI is one of the first P300-based assistive robotic devices with multiple targets evaluated on people with ALS. Second, we provide a database with calibration data and online EEG recordings obtained in the evaluation of our BMI. This data is useful to develop and compare other BMI systems and test the processing pipelines of similar applications.

[1]  Ricardo Caraza,et al.  A multidisciplinary clinic for amyotrophic lateral sclerosis patients in Northeast Mexico , 2020, Revista Mexicana de Neurociencia.

[2]  Dennis J. McFarland,et al.  P 300-based brain-computer interface ( BCI ) event-related potentials ( ERPs ) : People with amyotrophic lateral sclerosis ( ALS ) vs . age-matched controls , 2015 .

[3]  Javier Andreu-Perez,et al.  A P300-based brain computer interface for smart home interaction through an ANFIS ensemble , 2017, 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[4]  Kazunari Tanaka,et al.  Timely manner application of hand orthoses to patients with amyotrophic lateral sclerosis: A case report , 2014, Prosthetics and orthotics international.

[5]  Richard D. Deveaux,et al.  Applied Smoothing Techniques for Data Analysis , 1999, Technometrics.

[6]  Xingyu Wang,et al.  Exploring Combinations of Different Color and Facial Expression Stimuli for Gaze-Independent BCIs , 2016, Front. Comput. Neurosci..

[7]  Ji-Hoon Jeong,et al.  Classification of Hand Motions within EEG Signals for Non-Invasive BCI-Based Robot Hand Control , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[8]  B. Kopp,et al.  Neural correlates of cognitive set shifting in amyotrophic lateral sclerosis , 2016, Clinical Neurophysiology.

[9]  Susan M. Smith,et al.  Upper Extremity Orthoses Use in Amyotrophic Lateral Sclerosis/Motor Neuron Disease: Three Case Reports , 2014, Hand.

[10]  Omar Mendoza-Montoya Development of a Hybrid Brain-Computer Interface for Autonomous Systems , 2018 .

[11]  T. Vaughan Brain-computer interfaces for people with amyotrophic lateral sclerosis. , 2020, Handbook of clinical neurology.

[12]  Sonja C. Kleih,et al.  Toward a P300 Based Brain-Computer Interface for Aphasia Rehabilitation after Stroke: Presentation of Theoretical Considerations and a Pilot Feasibility Study , 2016, Front. Hum. Neurosci..

[13]  Febo Cincotti,et al.  Assistive device with conventional, alternative, and brain-computer interface inputs to enhance interaction with the environment for people with amyotrophic lateral sclerosis: a feasibility and usability study. , 2015, Archives of physical medicine and rehabilitation.

[14]  Tadeusz Mikolajczyk,et al.  Robotics for rehabilitation of hand movement in stroke survivors , 2019, Advances in Mechanical Engineering.

[15]  J. Wolpaw,et al.  P300-based brain-computer interface (BCI) event-related potentials (ERPs): People with amyotrophic lateral sclerosis (ALS) vs. age-matched controls , 2015, Clinical Neurophysiology.

[16]  Alina A. von Davier,et al.  Cross-Validation , 2014 .

[17]  F. Cincotti,et al.  Developing brain-computer interfaces from a user-centered perspective: Assessing the needs of persons with amyotrophic lateral sclerosis, caregivers, and professionals. , 2015, Applied ergonomics.

[18]  Yijun Wang,et al.  Discriminative Canonical Pattern Matching for Single-Trial Classification of ERP Components , 2019, IEEE Transactions on Biomedical Engineering.

[19]  Christiane,et al.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. , 2004, Journal international de bioethique = International journal of bioethics.

[20]  A. Cichocki,et al.  A survey of the dummy face and human face stimuli used in BCI paradigm , 2015, Journal of Neuroscience Methods.

[21]  K. Y. Tong,et al.  An EMG-driven exoskeleton hand robotic training device on chronic stroke subjects: Task training system for stroke rehabilitation , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[22]  Berdakh Abibullaev,et al.  Design and evaluation of a P300-ERP based BCI system for real-time control of a mobile robot , 2017, 2017 5th International Winter Conference on Brain-Computer Interface (BCI).

[23]  Antonio Cerasa,et al.  The corticospinal tract profile in amyotrophic lateral sclerosis , 2017, Human brain mapping.

[24]  M. Nilsson,et al.  The Soft Extra Muscle system for improving the grasping capability in neurological rehabilitation , 2012, 2012 IEEE-EMBS Conference on Biomedical Engineering and Sciences.

[25]  Jongmin Lee,et al.  Seeking RSVP Task Features Correlated with P300 Speller Performance , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[26]  A. Kagan The multidisciplinary clinic. , 2005, International journal of radiation oncology, biology, physics.

[27]  Goutam Chakraborty,et al.  Probes minimization still maintaining high accuracy to classify target stimuli P300 , 2017, 2017 IEEE 8th International Conference on Awareness Science and Technology (iCAST).

[28]  Daniela De Venuto,et al.  Real-time P300-based BCI in mechatronic control by using a multi-dimensional approach , 2018, IET Softw..

[29]  Berdakh Abibullaev,et al.  Design and evaluation of a P300 visual brain-computer interface speller in cyrillic characters , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[30]  Wolfgang Rosenstiel,et al.  Spatial Filtering Based on Canonical Correlation Analysis for Classification of Evoked or Event-Related Potentials in EEG Data , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  David J. Guggenmos,et al.  Current Challenges Facing the Translation of Brain Computer Interfaces from Preclinical Trials to Use in Human Patients , 2016, Front. Cell. Neurosci..

[32]  David Achanccaray,et al.  Performance Evaluation of a P300 Brain-Computer Interface Using a Kernel Extreme Learning Machine Classifier , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[33]  G. Riva,et al.  Cognitive assessment in Amyotrophic Lateral Sclerosis by means of P300-Brain Computer Interface: a preliminary study , 2016, Amyotrophic lateral sclerosis & frontotemporal degeneration.

[34]  Y. Shahriari,et al.  Neural Alterations During Use of a P300-based BCI by Individuals with Amyotrophic Lateral Sclerosis* , 2019, 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER).

[35]  Dennis J McFarland,et al.  Brain‐computer interfaces for amyotrophic lateral sclerosis , 2020, Muscle & nerve.

[36]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[37]  A. Cichocki,et al.  Comparison of the ERP-Based BCI Performance Among Chromatic (RGB) Semitransparent Face Patterns , 2020, Frontiers in Neuroscience.

[38]  Tetsuo Komori,et al.  Operation of a P300-based brain-computer interface by patients with spinocerebellar ataxia , 2017, Clinical neurophysiology practice.

[39]  K. Kansaku,et al.  Operation of a P300-based brain–computer interface by individuals with cervical spinal cord injury , 2011, Clinical Neurophysiology.

[40]  Yogendra P. Chaubey,et al.  Resampling Methods: A Practical Guide to Data Analysis , 2000, Technometrics.

[41]  Chang-Hwan Im,et al.  EEG-Based Brain-Computer Interfaces: A Thorough Literature Survey , 2013, Int. J. Hum. Comput. Interact..

[42]  Marco Congedo,et al.  A Comparison of Mobile VR Display Running on an Ordinary Smartphone With Standard PC Display for P300-BCI Stimulus Presentation , 2020, IEEE Transactions on Games.

[43]  Gianluca Antonelli,et al.  Assistive robot operated via P300-based brain computer interface , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Cuntai Guan,et al.  An Efficient P300-based Brain-Computer Interface with Minimal Calibration Time , 2009, NIPS 2009.

[45]  Tae-Seong Kim,et al.  A P300 brain computer interface based intelligent home control system using a random forest classifier , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[46]  Matthias Marx,et al.  A Shoulder-Surfing Resistant Image-Based Authentication Scheme with a Brain-Computer Interface , 2019, 2019 International Conference on Cyberworlds (CW).

[47]  Nicolae Alexandru Botezatu,et al.  Controlling a hand orthosis by means of P300-based brain computer interface , 2015, 2015 E-Health and Bioengineering Conference (EHB).

[48]  Daniela M. Witten,et al.  An Introduction to Statistical Learning: with Applications in R , 2013 .

[49]  Zongtan Zhou,et al.  A 3D Visual Stimuli Based P300 Brain-computer Interface: for a Robotic Arm Control , 2017, AIACT '17.

[50]  Théodore Papadopoulo,et al.  Brain computer interface with the P300 speller: Usability for disabled people with amyotrophic lateral sclerosis. , 2018, Annals of physical and rehabilitation medicine.

[51]  Andrea Kübler,et al.  Hearing the needs of clinical users. , 2020, Handbook of clinical neurology.

[52]  Eda Akman Aydin,et al.  P300-Based Asynchronous Brain Computer Interface for Environmental Control System , 2018, IEEE Journal of Biomedical and Health Informatics.

[53]  Shaoping Bai,et al.  WEXO: Smart Wheelchair Exoskeleton for ALS Patients , 2018 .

[54]  M. Molinari,et al.  Interfacing brain with computer to improve communication and rehabilitation after brain damage. , 2016, Progress in brain research.

[55]  Touradj Ebrahimi,et al.  An efficient P300-based brain–computer interface for disabled subjects , 2008, Journal of Neuroscience Methods.

[56]  T. Ichida World Medical Association declaration of Helsinki , 1991, Gastroenterologia Japonica.

[57]  Sungho Jo,et al.  P300-BCI-based authentication system , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[58]  Daniela De Venuto,et al.  Brain-computer interface using P300: a gaming approach for neurocognitive impairment diagnosis , 2016, 2016 IEEE International High Level Design Validation and Test Workshop (HLDVT).

[59]  Nader Pouratian,et al.  A comparison of stimulus types in online classification of the P300 speller using language models , 2017, PloS one.

[60]  J. Wolpaw,et al.  An exploration of BCI performance variations in people with amyotrophic lateral sclerosis using longitudinal EEG data , 2019, Journal of neural engineering.

[61]  Daniela De Venuto,et al.  An embedded system remotely driving mechanical devices by P300 brain activity , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[62]  Ariel J. Levine,et al.  Spatial organization of cortical and spinal neurons controlling motor behavior , 2012, Current Opinion in Neurobiology.

[63]  Wolzt,et al.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. , 2003, The Journal of the American College of Dentists.

[64]  Mohammad Bagher Menhaj,et al.  Control of a 2-DoF robotic arm using a P300-based brain-computer interface , 2019, ArXiv.

[65]  S. Yu. Gordleeva,et al.  The Changes of Corticospinal Excitability During the Control of Artificial Hand Through the Brain–Computer Interface Based on the P300 Component of Visual Evoked Potential , 2019, Human Physiology.