Developing a Motor Imagery-Based Real-Time Asynchronous Hybrid BCI Controller for a Lower-Limb Exoskeleton

This study aimed to develop an intuitive gait-related motor imagery (MI)-based hybrid brain-computer interface (BCI) controller for a lower-limb exoskeleton and investigate the feasibility of the controller under a practical scenario including stand-up, gait-forward, and sit-down. A filter bank common spatial pattern (FBCSP) and mutual information-based best individual feature (MIBIF) selection were used in the study to decode MI electroencephalogram (EEG) signals and extract a feature matrix as an input to the support vector machine (SVM) classifier. A successive eye-blink switch was sequentially combined with the EEG decoder in operating the lower-limb exoskeleton. Ten subjects demonstrated more than 80% accuracy in both offline (training) and online. All subjects successfully completed a gait task by wearing the lower-limb exoskeleton through the developed real-time BCI controller. The BCI controller achieved a time ratio of 1.45 compared with a manual smartwatch controller. The developed system can potentially be benefit people with neurological disorders who may have difficulties operating manual control.

[1]  Shyamanta M. Hazarika,et al.  Motor imagery and mental fatigue: inter-relationship and EEG based estimation , 2018, Journal of Computational Neuroscience.

[2]  Pierluigi Zoccolotti,et al.  Rehabilitative devices for a top-down approach , 2019, Expert review of medical devices.

[3]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[4]  Xinyu Wu,et al.  Implementation of a Brain-Computer Interface on a Lower-Limb Exoskeleton , 2018, IEEE Access.

[5]  Jin-Woo Jeong,et al.  Motor Imagery EEG Classification Using Capsule Networks† , 2019, Sensors.

[6]  Klaus-Robert Müller,et al.  Brain-Switches for Asynchronous Brain–Computer Interfaces: A Systematic Review , 2020 .

[7]  G. Pfurtscheller,et al.  Self-Paced Operation of an SSVEP-Based Orthosis With and Without an Imagery-Based “Brain Switch:” A Feasibility Study Towards a Hybrid BCI , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Hwi-young Cho,et al.  randomized controlled trial Effects of motor imagery training on balance and gait abilities in post-stroke patients : a , 2013 .

[9]  Renato Salinas,et al.  Recognition and Real-Time Detection of Blinking Eyes on Electroencephalographic Signals Using Wavelet Transform , 2012, CIARP.

[10]  Dewen Hu,et al.  An Asynchronous Hybrid Spelling Approach Based on EEG–EOG Signals for Chinese Character Input , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  Jianhua Wang,et al.  EEG-based brain-controlled lower extremity exoskeleton rehabilitation robot , 2017, 2017 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM).

[12]  Ian Daly,et al.  Bispectrum-Based Channel Selection for Motor Imagery Based Brain-Computer Interfacing , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[13]  G. Cheron,et al.  Oscillations in the human brain during walking execution, imagination and observation , 2015, Neuropsychologia.

[14]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[15]  F. Lacquaniti,et al.  From Spinal Central Pattern Generators to Cortical Network: Integrated BCI for Walking Rehabilitation , 2012, Neural plasticity.

[16]  G. Pfurtscheller,et al.  Graz-BCI: state of the art and clinical applications , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  Ian Daly,et al.  Internal Feature Selection Method of CSP Based on L1-Norm and Dempster–Shafer Theory , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Trieu Phat Luu,et al.  Brain–machine interfaces for controlling lower-limb powered robotic systems , 2018, Journal of neural engineering.

[19]  Shiqian Wang,et al.  Design and Control of the MINDWALKER Exoskeleton , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  Chang-Hwan Im,et al.  Development of a Brain–Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography , 2020, Sensors.

[21]  Ji-Hoon Jeong,et al.  Decoding of walking Intention under Lower limb exoskeleton Environment using MRCP Feature , 2017, GBCIC.

[22]  Milan Simic,et al.  Mu-Beta event-related (de)synchronization and EEG classification of left-right foot dorsiflexion kinaesthetic motor imagery for BCI , 2020, PloS one.

[23]  Tiago H. Falk,et al.  Deep learning-based electroencephalography analysis: a systematic review , 2019, Journal of neural engineering.

[24]  Ricardo Chavarriaga,et al.  Endogenous Control of Powered Lower-Limb Exoskeleton , 2017 .

[25]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[26]  José del R. Millán,et al.  Noninvasive brain-actuated control of a mobile robot by human EEG , 2004, IEEE Transactions on Biomedical Engineering.

[27]  G. Pfurtscheller,et al.  Continuous EEG classification during motor imagery-simulation of an asynchronous BCI , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[29]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[30]  S. Swinnen,et al.  Kinesthetic, but not visual, motor imagery modulates corticomotor excitability , 2005, Experimental Brain Research.

[31]  M. V. Lukoyanov,et al.  Exoskeleton Control System Based on Motor-Imaginary Brain–Computer Interface , 2017 .

[32]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[33]  M. Tsolaki,et al.  EEG-Based Brain–Computer Interfaces for Communication and Rehabilitation of People with Motor Impairment: A Novel Approach of the 21st Century , 2018, Front. Hum. Neurosci..

[34]  Jaeseung Jeong,et al.  Toward Brain-Actuated Humanoid Robots: Asynchronous Direct Control Using an EEG-Based BCI , 2012, IEEE Transactions on Robotics.

[35]  Manuel Rodriguez,et al.  Mu rhythm, visual processing and motor control , 2012, Clinical Neurophysiology.

[36]  D.J. McFarland,et al.  The wadsworth BCI research and development program: at home with BCI , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  M. Nicolelis,et al.  Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. , 2017, Physiological reviews.

[38]  Zoran Nenadic,et al.  Brain-computer interface controlled robotic gait orthosis , 2012, Journal of NeuroEngineering and Rehabilitation.

[39]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[40]  Milan Simic,et al.  EEG-Based BCI Control Schemes for Lower-Limb Assistive-Robots , 2018, Front. Hum. Neurosci..

[41]  W. A. Sarnacki,et al.  Brain–computer interface (BCI) operation: optimizing information transfer rates , 2003, Biological Psychology.

[42]  Jarrod A. Lewis-Peacock,et al.  Closed-loop brain training: the science of neurofeedback , 2017, Nature Reviews Neuroscience.

[43]  José M. Azorín,et al.  Sensory Integration in Human Movement: A New Brain-Machine Interface Based on Gamma Band and Attention Level for Controlling a Lower-Limb Exoskeleton , 2020, Frontiers in Bioengineering and Biotechnology.

[44]  Shouqian Sun,et al.  Single-trial EEG classification of motor imagery using deep convolutional neural networks , 2017 .

[45]  Justin C. Sanchez,et al.  DARPA-funded efforts in the development of novel brain–computer interface technologies , 2015, Journal of Neuroscience Methods.

[46]  Jaeheung Park,et al.  Development of actuation system for wearable robots using spiral spring , 2012, 2012 12th International Conference on Control, Automation and Systems.

[47]  Luis Montesano,et al.  Control of an Ambulatory Exoskeleton with a Brain–Machine Interface for Spinal Cord Injury Gait Rehabilitation , 2016, Front. Neurosci..

[48]  Klaus-Robert Müller,et al.  A lower limb exoskeleton control system based on steady state visual evoked potentials , 2015, Journal of neural engineering.

[49]  G. Cheron,et al.  Long-Lasting Cortical Reorganization as the Result of Motor Imagery of Throwing a Ball in a Virtual Tennis Court , 2015, Front. Psychol..

[50]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[51]  Hyungmin Kim,et al.  Robust Semi-synchronous BCI Controller for Brain-Actuated Exoskeleton System , 2020, 2020 8th International Winter Conference on Brain-Computer Interface (BCI).

[52]  Muhammad Abd-El-Barr,et al.  Long-term Training With a Brain-Machine Interface-Based Gait Protocol Induces Partial Neurological Recovery in Paraplegic Patients. , 2016, Neurosurgery.