Brain computer interface for gesture control of a social robot: An offline study

Brain computer interface (BCI) provides promising applications in neuroprosthesis and neurorehabilitation by controlling computers and robotic devices based on the patient's intentions. Here, we have developed a novel BCI platform that controls a personalized social robot using noninvasively acquired brain signals. Scalp electroencephalogram (EEG) signals are collected from a user in real-time during tasks of imaginary movements. The imagined body kinematics are decoded using a regression model to calculate the user-intended velocity. Then, the decoded kinematic information is mapped to control the gestures of a social robot. The platform here may be utilized as a human-robot-interaction framework by combining with neurofeedback mechanisms to enhance the cognitive capability of persons with dementia.

[1]  Richard J. Kryscio,et al.  Sugihara causality analysis of scalp EEG for detection of early Alzheimer's disease , 2014, NeuroImage: Clinical.

[2]  José L Contreras-Vidal,et al.  Decoding three-dimensional hand kinematics from electroencephalographic signals , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[3]  Baozeng Wang,et al.  Nonlinear EEG Decoding Based on a Particle Filter Model , 2014, BioMed research international.

[4]  Yoshiaki Hayashi,et al.  Motion Estimation Based on EMG and EEG Signals to Control Wearable Robots , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[5]  Trent J. Bradberry,et al.  Reconstructing Three-Dimensional Hand Movements from Noninvasive Electroencephalographic Signals , 2010, The Journal of Neuroscience.

[6]  Jaime Gómez Gil,et al.  Brain Computer Interfaces, a Review , 2012, Sensors.

[7]  Sungho Jo,et al.  A Low-Cost EEG System-Based Hybrid Brain-Computer Interface for Humanoid Robot Navigation and Recognition , 2013, PloS one.

[8]  Robert G. Grossman,et al.  NeuroRex: A clinical neural interface roadmap for EEG-based brain machine interfaces to a lower body robotic exoskeleton , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[9]  Wei Li,et al.  Development of Mind Control System for Humanoid Robot through a Brain Computer Interface , 2012, 2012 Second International Conference on Intelligent System Design and Engineering Application.

[10]  Yili Liu,et al.  EEG-Based Brain-Controlled Mobile Robots: A Survey , 2013, IEEE Transactions on Human-Machine Systems.

[11]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[12]  Rajesh P. N. Rao,et al.  Control of a humanoid robot by a noninvasive brain–computer interface in humans , 2008, Journal of neural engineering.

[13]  Richard J. Kryscio,et al.  Spectral and complexity analysis of scalp EEG characteristics for mild cognitive impairment and early Alzheimer's disease , 2014, Comput. Methods Programs Biomed..

[14]  N. Birbaumer,et al.  On the Usage of Linear Regression Models to Reconstruct Limb Kinematics from Low Frequency EEG Signals , 2013, PloS one.

[15]  Gernot R. Müller-Putz,et al.  Decoding of velocities and positions of 3D arm movement from EEG , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Bin He,et al.  Noninvasive control of a robotic arm in multiple dimensions using scalp electroencephalogram , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[17]  José Luis Contreras-Vidal,et al.  Observation-based training for neuroprosthetic control of grasping by amputees , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[19]  Christa Neuper,et al.  Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb , 2011, Medical & Biological Engineering & Computing.

[20]  G. Pfurtscheller,et al.  Self-Paced Operation of an SSVEP-Based Orthosis With and Without an Imagery-Based “Brain Switch:” A Feasibility Study Towards a Hybrid BCI , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[22]  M. Bergamasco,et al.  A New Gaze-BCI-Driven Control of an Upper Limb Exoskeleton for Rehabilitation in Real-World Tasks , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Y. Jiang,et al.  Scalp EEG-Based Discrimination of Cognitive Deficits After Traumatic Brain Injury Using Event-Related Tsallis Entropy Analysis , 2013, IEEE Transactions on Biomedical Engineering.

[25]  Xiaopeng Zhao,et al.  Resting EEG Discrimination of Early Stage Alzheimer’s Disease from Normal Aging Using Inter-Channel Coherence Network Graphs , 2013, Annals of Biomedical Engineering.

[26]  Jun Morimoto,et al.  Brain-machine interfacing control of whole-body humanoid motion , 2014, Front. Syst. Neurosci..

[27]  Genshe Chen,et al.  An adaptive P300 model for controlling a humanoid robot with mind , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[28]  Trent J. Bradberry,et al.  Fast attainment of computer cursor control with noninvasively acquired brain signals , 2011, Journal of neural engineering.

[29]  Gernot R. Müller-Putz,et al.  Control of an Electrical Prosthesis With an SSVEP-Based BCI , 2008, IEEE Transactions on Biomedical Engineering.

[30]  Brendan Z. Allison,et al.  Journal of Neuroscience Methods , 2022 .

[31]  Andrés Úbeda,et al.  Linear decoding of 2D hand movements for target selection tasks using a non-invasive BCI system , 2013, 2013 IEEE International Systems Conference (SysCon).

[32]  A. Graser,et al.  Low level control in a semi-autonomous rehabilitation robotic system via a Brain-Computer Interface , 2007, 2007 IEEE 10th International Conference on Rehabilitation Robotics.

[33]  Z. Gu,et al.  Decoding hand movement velocity from electroencephalogram signals during a drawing task , 2010, Biomedical engineering online.

[34]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[35]  Rajesh P. N. Rao,et al.  An adaptive brain-computer interface for humanoid robot control , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[36]  Ulrich G. Hofmann,et al.  Avatar navigation in Second Life using brain signals , 2013, 2013 IEEE 8th International Symposium on Intelligent Signal Processing.

[37]  Feng Guo,et al.  Brain–machine interface control of a manipulator using small-world neural network and shared control strategy , 2014, Journal of Neuroscience Methods.

[38]  Amit Konar,et al.  Implementation of EEG based control of remote robotic systems , 2011, 2011 International Conference on Recent Trends in Information Systems.

[39]  Andrew Y. Paek,et al.  Global cortical activity predicts shape of hand during grasping , 2015, Front. Neurosci..

[40]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[41]  Reza Abiri,et al.  A Real-Time Brainwave Based Neuro-Feedback System for Cognitive Enhancement , 2015 .

[42]  Jose L. Contreras-Vidal,et al.  A pre-clinical framework for neural control of a therapeutic upper-limb exoskeleton , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[43]  Jie Li,et al.  A combination strategy based brain–computer interface for two-dimensional movement control , 2015, Journal of neural engineering.