Real-Time Control of a Video Game Using Eye Movements and Two Temporal EEG Sensors

EEG-controlled gaming applications range widely from strictly medical to completely nonmedical applications. Games can provide not only entertainment but also strong motivation for practicing, thereby achieving better control with rehabilitation system. In this paper we present real-time control of video game with eye movements for asynchronous and noninvasive communication system using two temporal EEG sensors. We used wavelets to detect the instance of eye movement and time-series characteristics to distinguish between six classes of eye movement. A control interface was developed to test the proposed algorithm in real-time experiments with opened and closed eyes. Using visual feedback, a mean classification accuracy of 77.3% was obtained for control with six commands. And a mean classification accuracy of 80.2% was obtained using auditory feedback for control with five commands. The algorithm was then applied for controlling direction and speed of character movement in two-dimensional video game. Results showed that the proposed algorithm had an efficient response speed and timing with a bit rate of 30 bits/min, demonstrating its efficacy and robustness in real-time control.

[1]  Yasuharu Koike,et al.  Classification of Four Eye Directions from EEG Signals for Eye-Movement-Based Communication Systems , 2014 .

[2]  Sadık Kara,et al.  Classification of electro-oculogram signals using artificial neural network , 2006, Expert Syst. Appl..

[3]  Masaaki Fukumoto,et al.  Full-time wearable headphone-type gaze detector , 2006, CHI Extended Abstracts.

[4]  G Pfurtscheller,et al.  Toward a hybrid brain–computer interface based on imagined movement and visual attention , 2010, Journal of neural engineering.

[5]  M. Mazo,et al.  System for assisted mobility using eye movements based on electrooculography , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  John R. Smith,et al.  Steady-State VEP-Based Brain-Computer Interface Control in an Immersive 3D Gaming Environment , 2005, EURASIP J. Adv. Signal Process..

[7]  Yasuharu Koike,et al.  Online classification algorithm for eye-movement-based communication systems using two temporal EEG sensors , 2015, Biomed. Signal Process. Control..

[8]  Helge J. Ritter,et al.  2009 Special Issue: The MindGame: A P300-based brain-computer interface game , 2009 .

[9]  Aiguo Song,et al.  EOG Artifact Correction from EEG Recording Using Stationary Subspace Analysis and Empirical Mode Decomposition , 2013, Sensors.

[10]  Damien Coyle,et al.  Games, Gameplay, and BCI: The State of the Art , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[11]  Gary E. Birch,et al.  Online Removal of Eye Movement and Blink EEG Artifacts Using a High-Speed Eye Tracker , 2012, IEEE Transactions on Biomedical Engineering.

[12]  Yuanqing Li,et al.  A Hybrid Brain Computer Interface to Control the Direction and Speed of a Simulated or Real Wheelchair , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[13]  Doru Talaba,et al.  EOG-based visual navigation interface development , 2012, Expert Syst. Appl..

[14]  W W Abbott,et al.  Ultra-low-cost 3D gaze estimation: an intuitive high information throughput compliment to direct brain–machine interfaces , 2012, Journal of neural engineering.

[15]  Christa Neuper,et al.  Human Brain – Computer Interface , 2005 .

[16]  Dinesh Kumar,et al.  Classification of EOG for human computer interface , 2002, Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology.

[17]  G. Pfurtscheller,et al.  Self-Paced Operation of an SSVEP-Based Orthosis With and Without an Imagery-Based “Brain Switch:” A Feasibility Study Towards a Hybrid BCI , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Qiang Wang,et al.  Fractal dimension based neurofeedback in serious games , 2011, The Visual Computer.

[19]  Minoru Sasaki,et al.  Affine Transform to Reform Pixel Coordinates of EOG Signals for Controlling Robot Manipulators Using Gaze Motions , 2014, Sensors.

[20]  Eilon Vaadia,et al.  Motor Cortex in Voluntary Movements: A Distributed System for Distributed Functions , 2007 .

[21]  Luciano Boquete,et al.  EOG-based eye movements codification for human computer interaction , 2012, Expert Syst. Appl..

[22]  Junichi Hori,et al.  Development of EOG-Based Communication System Controlled by Eight-Directional Eye Movements , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[23]  Tohru Yagi,et al.  Conductive rubber electrodes for earphone-based eye gesture input interface , 2014, Personal and Ubiquitous Computing.

[24]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[25]  A Värri,et al.  The effect of small differences in electrode position on EOG signals: application to vigilance studies. , 1993, Electroencephalography and clinical neurophysiology.

[26]  Serkan Gurkan,et al.  Design of a Novel Efficient Human–Computer Interface: An Electrooculagram Based Virtual Keyboard , 2010, IEEE Transactions on Instrumentation and Measurement.

[27]  Christa Neuper,et al.  14 Human Brain-Computer Interface , 2005 .

[28]  Chun-Liang Hsu,et al.  EOG-based Human-Computer Interface system development , 2010, Expert Syst. Appl..

[29]  Yuanqing Li,et al.  An EEG-Based BCI System for 2-D Cursor Control by Combining Mu/Beta Rhythm and P300 Potential , 2010, IEEE Transactions on Biomedical Engineering.

[30]  Deok-Hwan Kim,et al.  EOG based eye movement measure of visual fatigue caused by 2D and 3D displays , 2012, Proceedings of 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics.

[31]  Michael F. Marmor,et al.  Studies on the stability of the clinical electro-oculogram , 2004, Documenta Ophthalmologica.

[32]  Klaus-Robert Müller,et al.  Playing Pinball with non-invasive BCI , 2008, NIPS.

[33]  Yuanqing Li,et al.  An asynchronous wheelchair control by hybrid EEG–EOG brain–computer interface , 2014, Cognitive Neurodynamics.

[34]  D. Borghetti,et al.  A low-cost interface for control of computer functions by means of eye movements , 2007, Comput. Biol. Medicine.

[35]  Naoaki Itakura,et al.  A new method for calculating eye movement displacement from AC coupled electro-oculographic signals in head mounted eye-gaze input interfaces , 2010, Biomed. Signal Process. Control..