A multiplayer online car racing virtual-reality game based on internet of brains

Abstract Development of brain-computer interface (BCI)-controlled virtual reality (VR) games has received increasing attention. Yet, the up-to-date BCI-VR systems were still based on one single BCI and a virtual environment (VE). In this paper, we propose and implement a novel BCI-controlled VR (BCI-VR) game based on a structure of internet of brains (IoB) allowing multiple players from different sites to play a car racing game online. Electroencephalographic (EEG) and electromyographic (EMG) signals from different sites’ BCIs are uploaded to a high-performance cloud server where the car-controlled algorithms are performed. During the online car racing period, the players mentally control the speeds of their chosen cars by means of concentration, and the concentration level can be adjusted by performing a mental arithmetic (MA) task with different levels of difficulty. Two linear and two nonlinear EEG features, including theta band power (BP), beta BP, Higuchi's fractal dimension (HFD), and Katz's FD (KFD), are used to transform the concentration level to speeds of four different cars. The players can also sensitively trigger the car in the VE to jump by performing a slight teeth-gritting task to generate easy-to-detect EMG signals. Six subjects participated in this study to test the performance of the proposed hybrid (EEG plus EMG) BCI-VR car racing game. The results indicate that theta BP and HFD are more sensitive to the MA-induced concentration in comparison with beta BP and KFD. Through the test of online car racing game, the results also demonstrated the feasibility that different players play the game in the same VE through multiple BCI control at different sites. More importantly, our BCI-VR implementation has a high usability (only two electrodes are required; calibration needs only 64 s) and high feasibility (high average scores of the control, sensory, and distraction factors in a 30-item post-experimental presence questionnaire).

[1]  Rosa H. M. Chan,et al.  An evaluation of mental workload with frontal EEG , 2017, PloS one.

[2]  Yi-Hung Liu,et al.  A Self-Paced P300 Healthcare Brain-Computer Interface System with SSVEP-Based Switching Control and Kernel FDA + SVM-Based Detector , 2016 .

[3]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[4]  Brian Litt,et al.  A comparison of waveform fractal dimension algorithms , 2001 .

[5]  R Beveridge,et al.  3D graphics, virtual reality, and motion-onset visual evoked potentials in neurogaming. , 2016, Progress in brain research.

[6]  Aurelio Tommasetti,et al.  A Review of Smart Cities Based on the Internet of Things Concept , 2017 .

[7]  Ulrich G. Hofmann,et al.  Avatar navigation in Second Life using brain signals , 2013, 2013 IEEE 8th International Symposium on Intelligent Signal Processing.

[8]  Yi-Hung Liu,et al.  Emotion Recognition from Single-Trial EEG Based on Kernel Fisher's Emotion Pattern and Imbalanced Quasiconformal Kernel Support Vector Machine , 2014, Sensors.

[9]  J. R. Wiersema,et al.  Resting electroencephalogram in attention deficit hyperactivity disorder: Developmental course and diagnostic value , 2014, Psychiatry Research.

[10]  Tao Chen,et al.  Intelligent fault prediction system based on internet of things , 2012, Comput. Math. Appl..

[11]  M. Thulasidas,et al.  Robust classification of EEG signal for brain-computer interface , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[13]  Aamir Saeed Malik,et al.  Electroencephalogram (EEG)-based computer-aided technique to diagnose major depressive disorder (MDD) , 2017, Biomed. Signal Process. Control..

[14]  Michitaka Hirose,et al.  Brain-Computer Interfaces, Virtual Reality, and Videogames , 2008, Computer.

[15]  Han-Pang Huang,et al.  Recognition of Electromyographic Signals Using Cascaded Kernel Learning Machine , 2007, IEEE/ASME Transactions on Mechatronics.

[16]  Anton Nijholt,et al.  Brain-Computer Interfaces Handbook: Technological and Theoretical Advances , 2018 .

[17]  M. J. Katz,et al.  Fractals and the analysis of waveforms. , 1988, Computers in biology and medicine.

[18]  Tao Liu,et al.  N200-speller using motion-onset visual response , 2009, Clinical Neurophysiology.

[19]  Nitish Thakor,et al.  Cognitive Workload Assessment Based on the Tensorial Treatment of EEG Estimates of Cross-Frequency Phase Interactions , 2014, Annals of Biomedical Engineering.

[20]  P. Grassberger,et al.  Measuring the Strangeness of Strange Attractors , 1983 .

[21]  Anatole Lécuyer,et al.  Exploring Large Virtual Environments by Thoughts Using a BrainComputer Interface Based on Motor Imagery and High-Level Commands , 2010, PRESENCE: Teleoperators and Virtual Environments.

[22]  Mel Slater,et al.  Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire , 1999, Presence.

[23]  Kazuhiro Shinosaki,et al.  The 3-dimensional representation of EEG distance by use of Shannon-Gelfand-Yaglom information measure during mental arithmetic , 2005, Brain Topography.

[24]  Yi-Hung Liu,et al.  Motor Imagery EEG Classification for Patients with Amyotrophic Lateral Sclerosis Using Fractal Dimension and Fisher’s Criterion-Based Channel Selection , 2017, Sensors.

[25]  Yi-Hung Liu,et al.  Major Depression Detection from EEG Signals Using Kernel Eigen-Filter-Bank Common Spatial Patterns , 2017, Sensors.

[26]  Yuval Bloch,et al.  Monitoring Attention in ADHD with an Easy-to-Use Electrophysiological Index , 2018, Front. Hum. Neurosci..

[27]  Q. Wang,et al.  Real-Time Mental Arithmetic Task Recognition From EEG Signals , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Ivo Käthner,et al.  Rapid P300 brain-computer interface communication with a head-mounted display , 2015, Front. Neurosci..

[29]  Kenneth Kwok,et al.  Evaluation Of Cognitive Workload From EEG During A Mental Arithmetic Task , 2011 .

[30]  G. Pfurtscheller,et al.  Brain–Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  Vera Kaiser,et al.  Thinking Penguin: Multimodal Brain–Computer Interface Control of a VR Game , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[32]  Francisco Velasco-Álvarez,et al.  Free Virtual Navigation Using Motor Imagery Through an Asynchronous BrainComputer Interface , 2010, PRESENCE: Teleoperators and Virtual Environments.

[33]  Yu-Te Wang,et al.  Pervasive brain monitoring and data sharing based on multi-tier distributed computing and linked data technology , 2014, Front. Hum. Neurosci..