The hybrid BCI system for movement control by combining motor imagery and moving onset visual evoked potential

OBJECTIVE Movement control is an important application for EEG-BCI (EEG-based brain-computer interface) systems. A single-modality BCI cannot provide an efficient and natural control strategy, but a hybrid BCI system that combines two or more different tasks can effectively overcome the drawbacks encountered in single-modality BCI control. APPROACH In the current paper, we developed a new hybrid BCI system by combining MI (motor imagery) and mVEP (motion-onset visual evoked potential), aiming to realize the more efficient 2D movement control of a cursor. MAIN RESULT The offline analysis demonstrates that the hybrid BCI system proposed in this paper could evoke the desired MI and mVEP signal features simultaneously, and both are very close to those evoked in the single-modality BCI task. Furthermore, the online 2D movement control experiment reveals that the proposed hybrid BCI system could provide more efficient and natural control commands. SIGNIFICANCE The proposed hybrid BCI system is compensative to realize efficient 2D movement control for a practical online system, especially for those situations in which P300 stimuli are not suitable to be applied.

[1]  Yuanqing Li,et al.  A Hybrid BCI System Combining P300 and SSVEP and Its Application to Wheelchair Control , 2013, IEEE Transactions on Biomedical Engineering.

[2]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[3]  Dewen Hu,et al.  Hybrid Brain-Computer Interface (BCI) based on the EEG and EOG signals. , 2014, Bio-medical materials and engineering.

[4]  G. Pfurtscheller,et al.  Event-related synchronization (ERS) in the alpha band--an electrophysiological correlate of cortical idling: a review. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[5]  Klaus-Robert Müller,et al.  The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials , 2004, IEEE Transactions on Biomedical Engineering.

[6]  Tao Liu,et al.  N200-speller using motion-onset visual response , 2009, Clinical Neurophysiology.

[7]  A. Geurts,et al.  Definition dependent properties of the cortical silent period in upper-extremity muscles, a methodological study , 2014, Journal of NeuroEngineering and Rehabilitation.

[8]  G. Pfurtscheller,et al.  Could the beta rebound in the EEG be suitable to realize a “brain switch”? , 2009, Clinical Neurophysiology.

[9]  J. Polich,et al.  P300 amplitude is determined by target-to-target interval. , 2002, Psychophysiology.

[10]  Tzyy-Ping Jung,et al.  High-speed spelling with a noninvasive brain–computer interface , 2015, Proceedings of the National Academy of Sciences.

[11]  Chiew Tong Lau,et al.  A New Discriminative Common Spatial Pattern Method for Motor Imagery Brain–Computer Interfaces , 2009, IEEE Transactions on Biomedical Engineering.

[12]  H. Spekreijse,et al.  Flicker and movement constituents of the pattern reversal response , 1985, Vision Research.

[13]  Rui Zhang,et al.  An Adaptive Motion-Onset VEP-Based Brain-Computer Interface , 2015, IEEE Transactions on Autonomous Mental Development.

[14]  G Pfurtscheller,et al.  Toward a hybrid brain–computer interface based on imagined movement and visual attention , 2010, Journal of neural engineering.

[15]  Andrzej Cichocki,et al.  Fully Online Multicommand Brain-Computer Interface with Visual Neurofeedback Using SSVEP Paradigm , 2007, Comput. Intell. Neurosci..

[16]  Rami Saab,et al.  A Hybrid Brain–Computer Interface Based on the Fusion of P300 and SSVEP Scores , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  R Thull,et al.  Vergleichende Untersuchungen zur Eignung eines neuen Oberflächenkonditionierungsverfahrens (Airsonic Mini Sandblaster®) in der Klebebrückentechnik / Comparative Studies on the Applicability of a New Surface Conditioning System (Airsonic Mini Sandblaster®) in Adhesive Bridging Technic , 2004, Biomedizinische Technik. Biomedical engineering.

[18]  G Pfurtscheller,et al.  Real-time EEG analysis with subject-specific spatial patterns for a brain-computer interface (BCI). , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[19]  P. Clarke,et al.  Comparison of visual evoked potentials to stationary and to moving patterns , 1973, Experimental Brain Research.

[20]  A. Kübler,et al.  Toward brain-computer interface based wheelchair control utilizing tactually-evoked event-related potentials , 2014, Journal of NeuroEngineering and Rehabilitation.

[21]  Yi Zhong,et al.  Increase in phosphorylation of PDK1 and cell survival after acute spinal cord injury , 2012, Journal of Neurological Sciences.

[22]  Yoshiaki Hayashi,et al.  Towards Hybrid EEG-EMG-Based Control Approaches to be Used in Bio-robotics Applications: Current Status, Challenges and Future Directions , 2013, Paladyn J. Behav. Robotics.

[23]  J. Wolpaw,et al.  A P300 event-related potential brain–computer interface (BCI): The effects of matrix size and inter stimulus interval on performance , 2006, Biological Psychology.

[24]  Reza Fazel-Rezai,et al.  A Review of Hybrid Brain-Computer Interface Systems , 2013, Adv. Hum. Comput. Interact..

[25]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[26]  F. H. Lopes da Silva,et al.  Short‐lived brain state after cued motor imagery in naive subjects , 2008, The European journal of neuroscience.

[27]  Xingyu Wang,et al.  A new hybrid BCI paradigm based on P300 and SSVEP , 2015, Journal of Neuroscience Methods.

[28]  V V Skvortsov,et al.  Transcranial Electrostimulation in Treatment of Chronic Diffuse Liver Diseases , 2015, Meditsinskaia tekhnika.

[29]  Yuanqing Li,et al.  Surfing the internet with a BCI mouse , 2012, Journal of neural engineering.

[30]  Umut Orhan,et al.  RSVP Keyboard TM : An EEG Based BCI Typing System with Context Information Fusion , 2013 .

[31]  Xiaorong Gao,et al.  A brain–computer interface using motion-onset visual evoked potential , 2008, Journal of neural engineering.

[32]  A. Cichocki,et al.  An optimized ERP brain–computer interface based on facial expression changes , 2014, Journal of neural engineering.

[33]  S. Page,et al.  Effects of mental practice on affected limb use and function in chronic stroke. , 2005, Archives of physical medicine and rehabilitation.

[34]  Nitish V. Thakor,et al.  Simultaneous Neural Control of Simple Reaching and Grasping With the Modular Prosthetic Limb Using Intracranial EEG , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Christa Neuper,et al.  Autocalibration and Recurrent Adaptation: Towards a Plug and Play Online ERD-BCI , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[36]  Christa Neuper,et al.  Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb , 2011, Medical & Biological Engineering & Computing.

[37]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[38]  D. Yao,et al.  Multiple Frequencies Sequential Coding for SSVEP-Based Brain-Computer Interface , 2012, PloS one.

[39]  T. Mulder.  Motor imagery and action observation: cognitive tools for rehabilitation , 2007, Journal of Neural Transmission.

[40]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[41]  D J McFarland,et al.  Brain-computer interface research at the Wadsworth Center. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[42]  Christa Neuper,et al.  Movement and ERD/ERS , 2003 .

[43]  S. McDonough,et al.  The adjunctive role of mental practice in the rehabilitation of the upper limb after hemiplegic stroke: a pilot studya , 2004, Clinical rehabilitation.

[44]  Shangkai Gao,et al.  An N200 speller integrating the spatial profile for the detection of the non-control state , 2012, Journal of neural engineering.

[45]  F. L. D. Silva,et al.  Beta rebound after different types of motor imagery in man , 2005, Neuroscience Letters.

[46]  Yuanqing Li,et al.  An EEG-Based BCI System for 2-D Cursor Control by Combining Mu/Beta Rhythm and P300 Potential , 2010, IEEE Transactions on Biomedical Engineering.

[47]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[48]  D. Yao,et al.  An Enhanced Probabilistic LDA for Multi-Class Brain Computer Interface , 2011, PloS one.

[49]  Xingyu Wang,et al.  A combined brain–computer interface based on P300 potentials and motion-onset visual evoked potentials , 2012, Journal of Neuroscience Methods.

[50]  Chetwyn C. H. Chan,et al.  Mental imagery for promoting relearning for people after stroke: a randomized controlled trial. , 2004, Archives of physical medicine and rehabilitation.

[51]  Yangsong Zhang,et al.  Z-Score Linear Discriminant Analysis for EEG Based Brain-Computer Interfaces , 2013, PloS one.

[52]  P. Derambure,et al.  Basic mechanisms of central rhythms reactivity to preparation and execution of a voluntary movement: a stereoelectroencephalographic study , 2003, Clinical Neurophysiology.

[53]  M. Kuba,et al.  Motion-onset VEPs: Characteristics, methods, and diagnostic use , 2007, Vision Research.

[54]  Abderrahmane Kheddar,et al.  SSVEP stimuli design for object-centric BCI , 2015 .

[55]  A. Guillot,et al.  Benefits of Motor Imagery Training on Muscle Strength , 2010, Journal of strength and conditioning research.

[56]  W. Byblow,et al.  Kinesthetic but not visual imagery assists in normalizing the CNV in Parkinson’s disease , 2006, Clinical Neurophysiology.

[57]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.