A hybrid BCI-controlled smart home system combining SSVEP and EMG for individuals with paralysis

Abstract In this study, electromyogram (EMG) signals associated with occlusal movement were integrated with steady-state visual evoked potentials (SSVEPs) to develop a hybrid brain–computer interface (hBCI)-based smart home control system for individuals with paralysis. The SSVEP paradigm was used to develop a system containing one main interface and five sub-interfaces corresponding to several devices during the working state, and one interface for the idle state. Participants controlled the devices by gazing at certain stimuli, which flickered at different frequencies for each function. Classical correlation analysis (CCA) of four channel EEG signals was used to recognize SSVEP features as intended selection. Several particular occlusal EMG patterns from the single channel of temporalis muscle were used to confirm the selected function, return from the sub-interface to the main interface, and switch the system on/off, respectively. Five healthy participants and five individuals with paralysis completed the system control experiment. The average target selection accuracy reached 97.5% and 83.6% in healthy participants and patients, while the confirmation accuracy in each group reached 97.6% and 96.9%, respectively. When SSVEPs were combined with EMG signals from occlusal movement to confirm the target selection, the actual control accuracy was maximized to 100%, and the information transmission rate (ITR) reached 45 bit/min among patients. Operation of the hBCI-based smart home control system did not cause higher mental or physical workload in patients compared to healthy participants. Our findings indicate that combining SSVEP and EMG signals effectively enhances the safety and interactivity of hBCI-based smart home systems.

[1]  Rui Zhang,et al.  A BCI-Based Environmental Control System for Patients With Severe Spinal Cord Injuries , 2017, IEEE Transactions on Biomedical Engineering.

[2]  Kansaku Kenji.  A BMI-based robotic exoskeleton for neurorehabilitation and daily actions: A hybrid control method using EMG and SSVEP , 2015 .

[3]  Brendan Z. Allison,et al.  Journal of Neuroscience Methods , 2022 .

[4]  Guanglin Li,et al.  Hybrid brain/muscle-actuated control of an intelligent wheelchair , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[5]  Feng Li,et al.  Discrimination Between Control and Idle States in Asynchronous SSVEP-Based Brain Switches: A Pseudo-Key-Based Approach , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  F Cincotti,et al.  Workload measurement in a communication application operated through a P300-based brain–computer interface , 2011, Journal of neural engineering.

[7]  Jzau-Sheng Lin,et al.  A Wireless BCI-Controlled Integration System in Smart Living Space for Patients , 2016, Wirel. Pers. Commun..

[8]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[9]  Xiaorong Gao,et al.  An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method , 2009, Journal of neural engineering.

[10]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[11]  Xingyu Wang,et al.  A new hybrid BCI paradigm based on P300 and SSVEP , 2015, Journal of Neuroscience Methods.

[12]  Mário Sarcinelli Filho,et al.  Proposal of a SSVEP-BCI to Command a Robotic Wheelchair , 2013 .

[13]  Momin Uppal,et al.  A Hybrid Training-Less Brain-Machine Interface Using SSVEP and EMG Signal , 2015, 2015 13th International Conference on Frontiers of Information Technology (FIT).

[14]  G Pfurtscheller,et al.  Self-initiation of EEG-based brain-computer communication using the heart rate response. , 2007, Journal of neural engineering.

[15]  Yili Liu,et al.  A speed and direction-based cursor control system with P300 and SSVEP , 2014, Biomed. Signal Process. Control..

[16]  Bo Hyeok Seo,et al.  Development of new brain computer interface based on EEG and EMG , 2009, 2008 IEEE International Conference on Robotics and Biomimetics.

[17]  Ronald M. Aarts,et al.  A Survey of Stimulation Methods Used in SSVEP-Based BCIs , 2010, Comput. Intell. Neurosci..

[18]  Brice Rebsamen,et al.  A brain controlled wheelchair to navigate in familiar environments. , 2010, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[19]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[20]  F Babiloni,et al.  P300-based brain–computer interface for environmental control: an asynchronous approach , 2011, Journal of neural engineering.

[21]  Gabriel Curio,et al.  Brain-computer communication and slow cortical potentials , 2004, IEEE Transactions on Biomedical Engineering.

[22]  Ingrid Wickelgren.  Brain-Computer Interface Adds a New Dimension , 2004, Science.

[23]  W. Donovan,et al.  International Standards For Neurological Classification Of Spinal Cord Injury , 2003, The journal of spinal cord medicine.

[24]  William Z Rymer,et al.  Guest Editorial Brain–Computer Interface Technology: A Review of the Second International Meeting , 2001 .

[25]  J. Huggins,et al.  Brain-computer interface: current and emerging rehabilitation applications. , 2015, Archives of physical medicine and rehabilitation.

[26]  Po-Lei Lee,et al.  Total Design of an FPGA-Based Brain–Computer Interface Control Hospital Bed Nursing System , 2013, IEEE Transactions on Industrial Electronics.

[27]  Jonathan R Wolpaw,et al.  Brain–computer interfaces as new brain output pathways , 2007, The Journal of physiology.

[28]  Xiaogang Chen,et al.  An online hybrid BCI system based on SSVEP and EMG , 2016, Journal of neural engineering.

[29]  Han-Jeong Hwang,et al.  Neurofeedback-based motor imagery training for brain–computer interface (BCI) , 2009, Journal of Neuroscience Methods.

[30]  Kiran George,et al.  Brain-controlled driving aid for electric wheelchairs , 2016, 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[31]  Xiaorong Gao,et al.  A high-ITR SSVEP-based BCI speller , 2014 .

[32]  Tzyy-Ping Jung,et al.  High-speed spelling with a noninvasive brain–computer interface , 2015, Proceedings of the National Academy of Sciences.

[33]  Niels Birbaumer,et al.  Abnormal Neural Connectivity in Schizophrenia and fMRI-Brain-Computer Interface as a Potential Therapeutic Approach , 2012, Front. Psychiatry.

[34]  G. Pfurtscheller,et al.  Self-Paced Operation of an SSVEP-Based Orthosis With and Without an Imagery-Based “Brain Switch:” A Feasibility Study Towards a Hybrid BCI , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Bertrand Rivet,et al.  Feasibility of BCI Control in a Realistic Smart Home Environment , 2016, Front. Hum. Neurosci..

[36]  A. Cichocki,et al.  Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives , 2010, Progress in Neurobiology.