Three-Dimensional Brain–Computer Interface Control Through Simultaneous Overt Spatial Attentional and Motor Imagery Tasks

Objective: While noninvasive electroenceph-alography (EEG) based brain-computer interfacing (BCI) has been successfully demonstrated in two-dimensional (2-D) control tasks, little work has been published regarding its extension to practical three-dimensional (3-D) control. Methods: In this study, we developed a new BCI approach for 3-D control by combining a novel form of endogenous visuospatial attentional modulation, defined as overt spatial attention (OSA), and motor imagery (MI). Results: OSA modulation was shown to provide comparable control to conventional MI modulation in both 1-D and 2-D tasks. Furthermore, this paper provides evidence for the functional independence of traditional MI and OSA, as well as an investigation into the simultaneous use of both. Using this newly proposed BCI paradigm, 16 participants successfully completed a 3-D eight-target control task. Nine of these subjects further demonstrated robust 3-D control in a 12-target task, significantly outperforming the information transfer rate achieved in the 1-D and 2-D control tasks (29.7 ± 1.6 b/min). Conclusion: These results strongly support the hypothesis that noninvasive EEG-based BCI can provide robust 3-D control through endogenous neural modulation in broader populations with limited training. Significance: Through the combination of the two strategies (MI and OSA), a substantial portion of the recruited subjects were capable of robustly controlling a virtual cursor in 3-D space. The proposed novel approach could broaden the dimensionality of BCI control and shorten the training time.

[1]  Yijun Wang,et al.  Visual and Auditory Brain–Computer Interfaces , 2014, IEEE Transactions on Biomedical Engineering.

[2]  Motoaki Kawanabe,et al.  Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information , 2014, NeuroImage.

[3]  Karim Jerbi,et al.  Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy , 2015, Journal of Neuroscience Methods.

[4]  L Tonin,et al.  An online EEG BCI based on covert visuospatial attention in absence of exogenous stimulation , 2013, Journal of neural engineering.

[5]  John J. Foxe,et al.  Increases in alpha oscillatory power reflect an active retinotopic mechanism for distracter suppression during sustained visuospatial attention. , 2006, Journal of neurophysiology.

[6]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[7]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[8]  Benjamin Blankertz,et al.  Towards a Cure for BCI Illiteracy , 2009, Brain Topography.

[9]  Christa Neuper,et al.  Future prospects of ERD/ERS in the context of brain-computer interface (BCI) developments. , 2006, Progress in brain research.

[10]  G. V. Simpson,et al.  Anticipatory Biasing of Visuospatial Attention Indexed by Retinotopically Specific α-Bank Electroencephalography Increases over Occipital Cortex , 2000, The Journal of Neuroscience.

[11]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[12]  José del R. Millán,et al.  Brain-Controlled Wheelchairs: A Robotic Architecture , 2013, IEEE Robotics & Automation Magazine.

[13]  Á. Pascual-Leone,et al.  α-Band Electroencephalographic Activity over Occipital Cortex Indexes Visuospatial Attention Bias and Predicts Visual Target Detection , 2006, The Journal of Neuroscience.

[14]  Manuel Schabus,et al.  A shift of visual spatial attention is selectively associated with human EEG alpha activity , 2005, The European journal of neuroscience.

[15]  Dennis J. McFarland,et al.  Brain-Computer Interface Operation of Robotic and Prosthetic Devices , 2008, Computer.

[16]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[17]  Bin He,et al.  Noninvasive Brain-Computer Interfaces Based on Sensorimotor Rhythms , 2015, Proceedings of the IEEE.

[18]  Chin-Teng Lin,et al.  A Real-Time Wireless Brain–Computer Interface System for Drowsiness Detection , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[19]  Chun-Hsiang Chuang,et al.  Wireless and Wearable EEG System for Evaluating Driver Vigilance , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[20]  T. Jung,et al.  Dry and Noncontact EEG Sensors for Mobile Brain–Computer Interfaces , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Rajesh P. N. Rao,et al.  Cortical activity during motor execution, motor imagery, and imagery-based online feedback , 2010, Proceedings of the National Academy of Sciences.

[22]  J. W. Minett,et al.  Optimizing the P300-based brain–computer interface: current status, limitations and future directions , 2011, Journal of neural engineering.

[23]  Febo Cincotti,et al.  Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond , 2015, Proceedings of the IEEE.

[24]  I..  THE ATTENTION SYSTEM OF THE HUMAN BRAIN , 2002 .

[25]  Brendan Z. Allison,et al.  Could Anyone Use a BCI? , 2010, Brain-Computer Interfaces.

[26]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[27]  Bin He,et al.  Exploring Cognitive Flexibility With a Noninvasive BCI Using Simultaneous Steady-State Visual Evoked Potentials and Sensorimotor Rhythms , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[29]  G. Pfurtscheller,et al.  ERD/ERS patterns reflecting sensorimotor activation and deactivation. , 2006, Progress in brain research.

[30]  Krishna V Shenoy,et al.  Human cortical prostheses: lost in translation? , 2009, Neurosurgical focus.

[31]  Yuanqing Li,et al.  An EEG-Based BCI System for 2-D Cursor Control by Combining Mu/Beta Rhythm and P300 Potential , 2010, IEEE Transactions on Biomedical Engineering.

[32]  Dong Ming,et al.  A hybrid BCI speller paradigm combining P300 potential and the SSVEP blocking feature , 2013, Journal of neural engineering.

[33]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[34]  G. Pfurtscheller,et al.  An SSVEP BCI to Control a Hand Orthosis for Persons With Tetraplegia , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Bin He,et al.  Brain–Computer Interfaces Using Sensorimotor Rhythms: Current State and Future Perspectives , 2014, IEEE Transactions on Biomedical Engineering.

[36]  Rajesh P. N. Rao,et al.  Towards adaptive classification for BCI , 2006, Journal of neural engineering.

[37]  Bin He,et al.  Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks , 2016, Scientific Reports.

[38]  Jianjun Meng,et al.  Selective Sensation Based Brain-Computer Interface via Mechanical Vibrotactile Stimulation , 2013, PloS one.

[39]  Ying Sun,et al.  Asynchronous P300 BCI: SSVEP-based control state detection , 2010, 2010 18th European Signal Processing Conference.

[40]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[41]  G. Pfurtscheller,et al.  Information transfer rate in a five-classes brain-computer interface , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[42]  Yue Zhao,et al.  A Wireless BCI and BMI System for Wearable Robots , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[43]  M. V. Gerven,et al.  Attention modulations of posterior alpha as a control signal for two-dimensional brain–computer interfaces , 2009, Journal of Neuroscience Methods.

[44]  C. Grozea,et al.  Bristle-sensors—low-cost flexible passive dry EEG electrodes for neurofeedback and BCI applications , 2011, Journal of neural engineering.

[45]  An H. Do,et al.  Brain-Computer Interface Controlled Functional Electrical Stimulation System for Ankle Movement , 2011, Journal of NeuroEngineering and Rehabilitation.

[46]  Mario Cortese,et al.  Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia , 2016, Science Robotics.

[47]  J R Wolpaw,et al.  Spatial filter selection for EEG-based communication. , 1997, Electroencephalography and clinical neurophysiology.

[48]  Rajesh P. N. Rao,et al.  Distributed cortical adaptation during learning of a brain–computer interface task , 2013, Proceedings of the National Academy of Sciences.

[49]  G. Pfurtscheller,et al.  Functional dissociation of lower and upper frequency mu rhythms in relation to voluntary limb movement , 2000, Clinical Neurophysiology.

[50]  Bin He,et al.  Negative covariation between task-related responses in alpha/beta-band activity and BOLD in human sensorimotor cortex: An EEG and fMRI study of motor imagery and movements , 2010, NeuroImage.

[51]  Tzyy-Ping Jung,et al.  High-speed spelling with a noninvasive brain–computer interface , 2015, Proceedings of the National Academy of Sciences.

[52]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[53]  Katherine M. Armstrong,et al.  Visuomotor Origins of Covert Spatial Attention , 2003, Neuron.

[54]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[55]  Stephen J. Anderson,et al.  Attentional modulation of oscillatory activity in human visual cortex , 2003, NeuroImage.

[56]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[57]  Ali Bahramisharif,et al.  Brain-computer interfacing using modulations of alpha activity induced by covert shifts of attention , 2011, Journal of NeuroEngineering and Rehabilitation.

[58]  José del R. Millán,et al.  Brain-Computer Interfaces , 2020, Handbook of Clinical Neurology.

[59]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.