An Electrooculogram-Based Interaction Method and Its Music-on-Demand Application in a Virtual Reality Environment

Recently, the devices commonly used for interaction with a virtual reality (VR) environment include game controllers, data gloves, and motion tracking systems. However, these devices may limit the immersive experience or cause inconvenience, especially for patients with limb paralysis. This paper proposes a new nonmanual human–computer interface (HCI) based on a single-channel electrooculogram (EOG) signal and enables real-time interactions with the VR environment. The graphical user interface of the EOG-based HCI in VR includes several buttons flashing in a random order. The user needs to blink in synchrony with the corresponding button’s flashes to issue a command, while the algorithm detects the eye blinks from the EOG signal and determines the users’ target button. Furthermore, with the EOG-based HCI, we developed a music-on-demand system in the VR environment. By blinking, users can search and select target songs in the music library, play the selected songs, switch to the next song, and delete songs from the list. Ten healthy subjects and five patients with spinal cord injuries (SCIs) participated in online experiments. The experimental results demonstrated the effectiveness of the EOG-based HCI as a new method for interacting with the VR environment. Furthermore, the music-on-demand system can be used for entertainment not only by healthy people but also by SCI patients with limb paralysis.

[1]  Á. Gil-Agudo,et al.  Virtual Reality System Toyra: A New Tool to Assess and Treatment for Upper Limb Motor Impairment in Patients with Spinal Cord Injury , 2013 .

[2]  Tao Zhang,et al.  Structural and functional correlates of motor imagery BCI performance: Insights from the patterns of fronto-parietal attention network , 2016, NeuroImage.

[3]  Yuanqing Li,et al.  An EOG-Based Human–Machine Interface for Wheelchair Control , 2018, IEEE Transactions on Biomedical Engineering.

[4]  Zachary Wartell,et al.  Leveraging change blindness for redirection in virtual environments , 2011, 2011 IEEE Virtual Reality Conference.

[5]  J. Loomis,et al.  Body-based senses enhance knowledge of directions in large-scale environments , 2004, Psychonomic bulletin & review.

[6]  Yuanqing Li,et al.  A Single-Channel EOG-Based Speller , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  R. Krauzlis.  The Control of Voluntary Eye Movements: New Perspectives , 2005, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[8]  Fumitoshi Matsuno,et al.  A Novel EOG/EEG Hybrid Human–Machine Interface Adopting Eye Movements and ERPs: Application to Robot Control , 2015, IEEE Transactions on Biomedical Engineering.

[9]  G Calhoun,et al.  Brain-computer interfaces based on the steady-state visual-evoked response. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[10]  Gerd Bruder,et al.  Tuning Self-Motion Perception in Virtual Reality with Visual Illusions , 2012, IEEE Transactions on Visualization and Computer Graphics.

[11]  Christian F. Doeller,et al.  Evidence for grid cells in a human memory network , 2010, Nature.

[12]  Jack M. Loomis,et al.  Using virtual environments to assess directional knowledge , 2004 .

[13]  P. Harris.  The International Standards Booklet for Neurological and Functional Classification of Spinal Cord Injury , 1994, Paraplegia.

[14]  K. Sunnerhagen,et al.  Assessment and Training in a 3-Dimensional Virtual Environment With Haptics: A Report on 5 Cases of Motor Rehabilitation in the Chronic Stage After Stroke , 2007, Neurorehabilitation and neural repair.

[15]  Yeung Sam Hung,et al.  Efficient Implementation and Design of a New Single-Channel Electrooculography-Based Human–Machine Interface System , 2015, IEEE Transactions on Circuits and Systems II: Express Briefs.

[16]  Robert V. Kenyon,et al.  Reaching Within a Dynamic Virtual Environment , 2006 .

[17]  Chun-Liang Hsu,et al.  EOG-based Human-Computer Interface system development , 2010, Expert Syst. Appl..

[18]  Bin He,et al.  Brain-Computer Interface Control in a Virtual Reality Environment and Applications for the Internet of Things , 2018, IEEE Access.

[19]  Jiyoung Kang.  Effect of Interaction Based on Augmented Context in Immersive Virtual Reality Environment , 2018, Wirel. Pers. Commun..

[20]  Tommy Strandvall,et al.  Eye Tracking in Human-Computer Interaction and Usability Research , 2009, INTERACT.

[21]  P. Verschure,et al.  Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation , 2010, Journal of NeuroEngineering and Rehabilitation.

[22]  Cuntai Guan,et al.  Asynchronous P300-Based Brain--Computer Interfaces: A Computational Approach With Statistical Models , 2008, IEEE Transactions on Biomedical Engineering.

[23]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[24]  Albrecht Schmidt,et al.  Advances in Tangible Interaction and Ubiquitous Virtual Reality , 2008, IEEE Pervasive Computing.

[25]  Antonio Frisoli,et al.  Local and Remote Cooperation With Virtual and Robotic Agents: A P300 BCI Study in Healthy and People Living With Spinal Cord Injury , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  G. Lewis,et al.  Virtual reality games for rehabilitation of people with stroke: perspectives from the users , 2011, Disability and rehabilitation. Assistive technology.

[27]  Hua Cai,et al.  An integrated head pose and eye gaze tracking approach to non-intrusive visual attention measurement for wide FOV simulators , 2010, Virtual Reality.

[28]  Christa Neuper,et al.  Walking by Thinking: The Brainwaves Are Crucial, Not the Muscles! , 2006, PRESENCE: Teleoperators and Virtual Environments.

[29]  Serkan Gurkan,et al.  Design of a Novel Efficient Human–Computer Interface: An Electrooculagram Based Virtual Keyboard , 2010, IEEE Transactions on Instrumentation and Measurement.

[30]  Michitaka Hirose,et al.  Brain-Computer Interfaces, Virtual Reality, and Videogames , 2008, Computer.

[31]  J. Wolpaw,et al.  A P300 event-related potential brain–computer interface (BCI): The effects of matrix size and inter stimulus interval on performance , 2006, Biological Psychology.

[32]  Amit Sharma,et al.  Electrooculogram-based virtual reality game control using blink detection and gaze calibration , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).