A survey on robots controlled by motor imagery brain-computer interfaces

Abstract A brain-computer interface (BCI) can provide a communication approach conveying brain information to the outside. Especially, the BCIs based on motor imagery play the important role for the brain-controlled robots, such as the rehabilitation robots, the wheelchair robots, the nursing bed robots, the unmanned aerial vehicles and so on. In this paper, the developments of the robots based on motor imagery BCIs are reviewed from three aspects: the electroencephalogram (EEG) evocation paradigms, the signal processing algorithms and the applications. First, the different types of the brain-controlled robots are reviewed and classified from the perspective of the evocation paradigms. Second, the relevant algorithms for the EEG signal processing are introduced, which including feature extraction methods and the classification algorithms. Third, the applications of the motor imagery brain-controlled robots are summarized. Finally, the current challenges and the future research directions of the robots controlled by the motor imagery BCIs are discussed.

[1]  Jaeseung Jeong,et al.  Noninvasive Brain-Computer Interface-based control of humanoid navigation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Pei-Yi Lin,et al.  Assessments of Muscle Oxygenation and Cortical Activity Using Functional Near-infrared Spectroscopy in Healthy Adults During Hybrid Activation , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Fakhita Regragui,et al.  EEG efficient classification of imagined right and left hand movement using RBF kernel SVM and the joint CWT_PCA , 2017, AI & SOCIETY.

[4]  K.W.E. Cheng,et al.  Towards a Brain-Computer Interface based control for next generation electric wheelchairs , 2009, 2009 3rd International Conference on Power Electronics Systems and Applications (PESA).

[5]  Renquan Lu,et al.  Development and Learning Control of a Human Limb With a Rehabilitation Exoskeleton , 2014, IEEE Transactions on Industrial Electronics.

[6]  Shingo Shimoda,et al.  A Synergetic Brain-Machine Interfacing Paradigm for Multi-DOF Robot Control , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[7]  Kazuo Tanaka,et al.  Electroencephalogram-based control of an electric wheelchair , 2005, IEEE Transactions on Robotics.

[8]  Sadasivan Puthusserypady,et al.  A Hybrid MI-SSVEP based Brain Computer Interface for Potential Upper Limb Neurorehabilitation: A Pilot Study , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).

[9]  Desney S. Tan,et al.  Brain-Computer Interfacing for Intelligent Systems , 2008, IEEE Intelligent Systems.

[10]  Jinming Li,et al.  Quadcopter Control System Using a Hybrid BCI Based on Off-Line Optimization and Enhanced Human-Machine Interaction , 2020, IEEE Access.

[11]  G. Pfurtscheller,et al.  Self-Paced Operation of an SSVEP-Based Orthosis With and Without an Imagery-Based “Brain Switch:” A Feasibility Study Towards a Hybrid BCI , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  Keum-Shik Hong,et al.  Hybrid EEG-NIRS based BCI for quadcopter control , 2015, 2015 54th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE).

[13]  Kyuwan Choi,et al.  Control of a Wheelchair by Motor Imagery in Real Time , 2008, IDEAL.

[14]  Amit Konar,et al.  Motor imagery and error related potential induced position control of a robotic arm , 2017, IEEE/CAA Journal of Automatica Sinica.

[15]  Amit Konar,et al.  Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose , 2014, Medical & Biological Engineering & Computing.

[16]  Vijayan K. Asari,et al.  Electroencephelograph based brain machine interface for controlling a robotic arm , 2013, 2013 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[17]  Jaeseung Jeong,et al.  Brain-actuated humanoid robot navigation control using asynchronous Brain-Computer Interface , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[18]  Kyuwan Choi.  Control of a vehicle with EEG signals in real-time and system evaluation , 2011, European Journal of Applied Physiology.

[19]  Jie Li,et al.  A hybrid brain computer interface system based on the neurophysiological protocol and brain-actuated switch for wheelchair control , 2014, Journal of Neuroscience Methods.

[20]  Xinjun Sheng,et al.  A Stimulus-Independent Hybrid BCI Based on Motor Imagery and Somatosensory Attentional Orientation , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Chun-Yi Su,et al.  Brain–Machine Interface and Visual Compressive Sensing-Based Teleoperation Control of an Exoskeleton Robot , 2017, IEEE Transactions on Fuzzy Systems.

[22]  S. N. Omkar,et al.  A Performance Study of 14-Channel and 5-Channel EEG Systems for Real-Time Control of Unmanned Aerial Vehicles (UAVs) , 2018, 2018 Second IEEE International Conference on Robotic Computing (IRC).

[23]  Jie Li,et al.  The research for the correlation between ERD/ERS and CSP , 2011, 2011 Seventh International Conference on Natural Computation.

[24]  Kabmun Cha,et al.  Hybrid MI-SSSEP Paradigm for classifying left and right movement toward BCI for exoskeleton control , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).

[25]  David J. Reinkensmeyer,et al.  Movement Anticipation and EEG: Implications for BCI-Contingent Robot Therapy , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Filippo Zappasodi,et al.  Deep learning for hybrid EEG-fNIRS brain–computer interface: application to motor imagery classification , 2018, Journal of neural engineering.

[27]  Bertrand Rivet,et al.  Drone, your brain, ring course: accept the challenge and prevail! , 2014, UbiComp Adjunct.

[28]  Yuanqing Li,et al.  Target Selection With Hybrid Feature for BCI-Based 2-D Cursor Control , 2012, IEEE Transactions on Biomedical Engineering.

[29]  Piotr Wolszczak,et al.  Construction of neural nets in brain-computer interface for robot arm steering , 2016, 2016 9th International Conference on Human System Interactions (HSI).

[30]  Enrique Hortal,et al.  Using a brain-machine interface to control a hybrid upper limb exoskeleton during rehabilitation of patients with neurological conditions , 2015, Journal of NeuroEngineering and Rehabilitation.

[31]  Yifan Wu,et al.  A New Brain-Computer Interface Paradigm based on Steady-State Visual Evoked Potential of Illusory Pattern Motion Perception* , 2019, 2019 16th International Conference on Ubiquitous Robots (UR).

[32]  G. Prasad,et al.  Applying a brain-computer interface to support motor imagery practice in people with stroke for upper limb recovery: a feasibility study , 2010, Journal of NeuroEngineering and Rehabilitation.

[33]  Yingzi Lin,et al.  Mobile Robot Control by BCI Based on Motor Imagery , 2014, 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics.

[34]  Atulya K. Nagar,et al.  A hybrid brain-computer interface for closed-loop position control of a robot arm , 2020, IEEE/CAA Journal of Automatica Sinica.

[35]  Howida A. Shedeed,et al.  Brain EEG signal processing for controlling a robotic arm , 2013, 2013 8th International Conference on Computer Engineering & Systems (ICCES).

[36]  M. Moghavvemi,et al.  Development of a steady state visual evoked potential (SSVEP)-based brain computer interface (BCI) system , 2007, 2007 International Conference on Intelligent and Advanced Systems.

[37]  Sungho Jo,et al.  Application of Hybrid Brain-Computer Interface with Augmented Reality on Quadcopter Control , 2020, 2020 8th International Winter Conference on Brain-Computer Interface (BCI).

[38]  Cuntai Guan,et al.  Towards improvement of MI-BCI performance of subjects with BCI deficiency , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[39]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[40]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[41]  Jianjun Meng,et al.  Selective Sensation Based Brain-Computer Interface via Mechanical Vibrotactile Stimulation , 2013, PloS one.

[42]  Dong Liu,et al.  A brain-controlled exoskeleton with cascaded event-related desynchronization classifiers , 2017, Robotics Auton. Syst..

[43]  Martin Spüler,et al.  Spatial filtering of EEG as a Regression Problem , 2017, GBCIC.

[44]  G. Pfurtscheller,et al.  Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[45]  Genshe Chen,et al.  Progress in EEG-Based Brain Robot Interaction Systems , 2017, Comput. Intell. Neurosci..

[46]  Jaeseung Jeong,et al.  Toward Brain-Actuated Humanoid Robots: Asynchronous Direct Control Using an EEG-Based BCI , 2012, IEEE Transactions on Robotics.

[47]  Chunfang Liu,et al.  A hybrid EEG-based BCI for robot grasp controlling , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[48]  Alex Alexandridis,et al.  Robot Motion Control via an EEG-Based Brain–Computer Interface by Using Neural Networks and Alpha Brainwaves , 2019, Electronics.

[49]  Abdullah Akce,et al.  Remote teleoperation of an unmanned aircraft with a brain-machine interface: Theory and preliminary results , 2010, 2010 IEEE International Conference on Robotics and Automation.

[50]  Qingsong Ai,et al.  Feature extraction of four-class motor imagery EEG signals based on functional brain network , 2019, Journal of neural engineering.

[51]  Mohammed Imamul Hassan Bhuiyan,et al.  Identification of motor imagery movements from EEG signals using Dual Tree Complex Wavelet Transform , 2015, 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[52]  Nicola Smania,et al.  Quantification of Upper Limb Motor Recovery and EEG Power Changes after Robot-Assisted Bilateral Arm Training in Chronic Stroke Patients: A Prospective Pilot Study , 2018, Neural plasticity.

[53]  Rihab Bousseta,et al.  EEG Based Brain Computer Interface for Controlling a Robot Arm Movement Through Thought , 2018 .

[54]  Saeid Nahavandi,et al.  Multiclass Informative Instance Transfer Learning Framework for Motor Imagery-Based Brain-Computer Interface , 2018, Comput. Intell. Neurosci..

[55]  Min Han,et al.  EEG signal classification for epilepsy diagnosis based on AR model and RVM , 2010, 2010 International Conference on Intelligent Control and Information Processing.

[56]  Gernot R. Müller-Putz,et al.  Functional Rehabilitation of the Paralyzed Upper Extremity After Spinal Cord Injury by Noninvasive Hybrid Neuroprostheses , 2015, Proceedings of the IEEE.

[57]  Zhihua Wang,et al.  Self-Paced Operation of a Wheelchair Based on a Hybrid Brain-Computer Interface Combining Motor Imagery and P300 Potential , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[58]  Tianwei Shi,et al.  Brain Computer Interface system based on indoor semi-autonomous navigation and motor imagery for Unmanned Aerial Vehicle control , 2015, Expert Syst. Appl..

[59]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[60]  Kee-Eung Kim,et al.  A POMDP Approach to Optimizing P300 Speller BCI Paradigm , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[61]  Saeid Nahavandi,et al.  A fresh look at functional link neural network for motor imagery-based brain–computer interface , 2018, Journal of Neuroscience Methods.

[62]  Ridha Djemal,et al.  Robot Navigation Using a Brain Computer Interface Based on Motor Imagery , 2018, Journal of Medical and Biological Engineering.

[63]  Kyung-Yoon Kim,et al.  Neurofeedback training improves the dual-task performance ability in stroke patients. , 2015, The Tohoku journal of experimental medicine.

[64]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[65]  Chenguang Yang,et al.  Mind guided motion control of robot manipulator using EEG signals , 2015, 2015 5th International Conference on Information Science and Technology (ICIST).

[66]  B. Allison,et al.  Paired Associative Stimulation Using Brain-Computer Interfaces for Stroke Rehabilitation: A Pilot Study , 2016, European journal of translational myology.

[67]  Jessica Cantillo-Negrete,et al.  Motor Imagery-Based Brain-Computer Interface Coupled to a Robotic Hand Orthosis Aimed for Neurorehabilitation of Stroke Patients , 2018, Journal of healthcare engineering.

[68]  Christa Neuper,et al.  Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb , 2011, Medical & Biological Engineering & Computing.

[69]  Maria José Blanca-Mena,et al.  Brain-Computer Interface application: auditory serial interface to control a two-class motor-imagery-based wheelchair , 2017, Journal of NeuroEngineering and Rehabilitation.

[70]  Mahdi Tavakoli,et al.  Assistive technology design and preliminary testing of a robot platform based on movement intention using low-cost brain computer interface , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[71]  Qingsong Ai,et al.  Feature Selection for Motor Imagery EEG Classification Based on Firefly Algorithm and Learning Automata , 2017, Sensors.

[72]  Thorsten O. Zander,et al.  A Survey on Unmanned Aerial Vehicle Remote Control Using Brain–Computer Interface , 2018, IEEE Transactions on Human-Machine Systems.

[73]  T. Satya Savithri,et al.  Autonomuos robot control based on EEG and cross-correlation , 2016, 2016 10th International Conference on Intelligent Systems and Control (ISCO).

[74]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[75]  V. Krajča,et al.  P03-Dimension reduction of EEG feature space by using PCA , 2018, Clinical Neurophysiology.

[76]  Ji-Hoon Jeong,et al.  Trajectory Decoding of Arm Reaching Movement Imageries for Brain-Controlled Robot Arm System , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[77]  Qiang Gao,et al.  Noninvasive Electroencephalogram Based Control of a Robotic Arm for Writing Task Using Hybrid BCI System , 2017, BioMed research international.

[78]  B. B. Zaidan,et al.  A review of disability EEG based wheelchair control system: Coherent taxonomy, open challenges and recommendations , 2018, Comput. Methods Programs Biomed..

[79]  Ridha Djemal,et al.  Comprehensive review on brain-controlled mobile robots and robotic arms based on electroencephalography signals , 2020, Intell. Serv. Robotics.

[80]  Vicente Alarcón Aquino,et al.  A BCI motor imagery experiment based on parametric feature extraction and Fisher Criterion , 2012, CONIELECOMP 2012, 22nd International Conference on Electrical Communications and Computers.

[81]  Bradford J McFadyen,et al.  Efficacy of virtual reality-based intervention on balance and mobility disorders post-stroke: a scoping review , 2015, Journal of NeuroEngineering and Rehabilitation.

[82]  Mohammad Ali Badamchizadeh,et al.  EEG Artifacts Handling in a Real Practical Brain–Computer Interface Controlled Vehicle , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[83]  Chung-Chuan Cheng,et al.  An automatic segmentation and classification framework for anti-nuclear antibody images , 2013, BioMedical Engineering OnLine.

[84]  Brice Rebsamen,et al.  A brain controlled wheelchair to navigate in familiar environments. , 2010, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[85]  Saeid Nahavandi,et al.  Towards automated quality assessment measure for EEG signals , 2017, Neurocomputing.

[86]  Yiming Deng,et al.  Tractor Assistant Driving Control Method Based on EEG Combined With RNN-TL Deep Learning Algorithm , 2020, IEEE Access.

[87]  Xinjun Sheng,et al.  Shared control of a robotic arm using non-invasive brain-computer interface and computer vision guidance , 2019, Robotics Auton. Syst..

[88]  Guangming Shi,et al.  Motor-Imagery-Based Teleoperation of a Dual-Arm Robot Performing Manipulation Tasks , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[89]  D. N. Tibarewala,et al.  EEG controlled remote robotic system from motor imagery classification , 2012, 2012 Third International Conference on Computing, Communication and Networking Technologies (ICCCNT'12).

[90]  Yunfa Fu,et al.  A Novel Deep Learning Scheme for Motor Imagery EEG Decoding Based on Spatial Representation Fusion , 2020, IEEE Access.

[91]  Jianhua Wang,et al.  A brain-controlled lower-limb exoskeleton for human gait training. , 2017, The Review of scientific instruments.

[92]  Clemens Brunner,et al.  Variability of ICA decomposition may impact EEG signals when used to remove eyeblink artifacts. , 2017, Psychophysiology.

[93]  Ashraf Saleem,et al.  Design of a brain controlled hand exoskeleton for patients with motor neuron diseases , 2015, 2015 10th International Symposium on Mechatronics and its Applications (ISMA).

[94]  Yili Liu,et al.  EEG-Based Brain-Controlled Mobile Robots: A Survey , 2013, IEEE Transactions on Human-Machine Systems.

[95]  Cuntai Guan,et al.  A Randomized Controlled Trial of EEG-Based Motor Imagery Brain-Computer Interface Robotic Rehabilitation for Stroke , 2015, Clinical EEG and neuroscience.

[96]  Brain Computer Interfaces, Principles and Practise , 2013 .

[97]  Yangsong Zhang,et al.  Multivariate synchronization index for frequency recognition of SSVEP-based brain–computer interface , 2014, Journal of Neuroscience Methods.

[98]  Moncef Gabbouj,et al.  Epileptic Seizure Classification of EEG Time-Series Using Rational Discrete Short-Time Fourier Transform , 2015, IEEE Transactions on Biomedical Engineering.

[99]  Álvaro Fernández-Rodríguez,et al.  Wheelchair navigation with an audio-cued, two-class motor imagery-based brain-computer interface system , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[100]  Surjo R. Soekadar,et al.  Sensory Feedback with a Hand Exoskeleton Increases EEG Modulation in a Brain-Machine Interface System , 2018, Converging Clinical and Engineering Research on Neurorehabilitation III.

[101]  Xingyu Wang,et al.  Sparse Bayesian Learning for Obtaining Sparsity of EEG Frequency Bands Based Feature Vectors in Motor Imagery Classification , 2017, Int. J. Neural Syst..

[102]  Bin He,et al.  Brain–Computer Interfaces Using Sensorimotor Rhythms: Current State and Future Perspectives , 2014, IEEE Transactions on Biomedical Engineering.

[103]  MengChu Zhou,et al.  Common Bayesian Network for Classification of EEG-Based Multiclass Motor Imagery BCI , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[104]  Qingsong Ai,et al.  Brain-robot Shared Control Based on Motor Imagery and Improved Bayes Filter* , 2019, 2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).

[105]  Amit Konar,et al.  Interval type-2 fuzzy logic based multiclass ANFIS algorithm for real-time EEG based movement control of a robot arm , 2015, Robotics Auton. Syst..

[106]  Mohammad H. Alomari,et al.  Automated Classification of L/R Hand Movement EEG Signals using Advanced Feature Extraction and Machine Learning , 2013, ArXiv.

[107]  Jinwei Sun,et al.  Removal of ocular artifacts using ICA and adaptive filter for motor imagery-based BCI , 2017 .

[108]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[109]  Juan Li,et al.  Study of A Brain-Controlled Switch during Motor Imagery , 2018, 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM).