Combining BCI with Virtual Reality: Towards New Applications and Improved BCI

Brain–Computer Interfaces (BCI) are communication systems which can convey messages through brain activity alone. Recently BCIs were gaining interest among the virtual reality (VR) community since they have appeared as promising interaction devices for virtual environments (VEs). Especially these implicit interaction techniques are of great interest for the VR community, e.g., you are imaging the movement of your hand and the virtual hand is moving, or you can navigate through houses or museums by your thoughts alone or just by looking at some highlighted objects. Furthermore, VE can provide an excellent testing ground for procedures that could be adapted to real world scenarios, especially patients with disabilities can learn to control their movements or perform specific tasks in a VE. Several studies will highlight these interactions.

[1]  Reinhold Scherer,et al.  Combining BCI and Virtual Reality: Scouting Virtual Worlds , 2007 .

[2]  Gernot R. Müller-Putz,et al.  Self-Paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A Case Study with a Tetraplegic , 2007, Comput. Intell. Neurosci..

[3]  Doron Friedman,et al.  Neurophysiology-based art in immersive virtual reality , 2009, Int. J. Arts Technol..

[4]  Grigore C. Burdea,et al.  Force and Touch Feedback for Virtual Reality , 1996 .

[5]  Guillaume Gibert,et al.  OpenViBE: An Open-Source Software Platform to Design, Test, and Use BrainComputer Interfaces in Real and Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[6]  G Pfurtscheller,et al.  Cardiac responses induced during thought-based control of a virtual environment. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[7]  J.D. Bayliss,et al.  Use of the evoked potential P3 component for control in a virtual apartment , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  M. Slater,et al.  Control of a Smart Home with a Brain-Computer Interface , 2008 .

[9]  Reza Fazel-Rezai,et al.  Recent Advances in Brain-Computer Interface Systems , 2011 .

[10]  Anton Nijholt,et al.  Turning Shortcomings into Challenges: Brain-Computer Interfaces for Games , 2009, INTETAIN.

[11]  G. Pfurtscheller,et al.  Brain–Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  G. Pfurtscheller,et al.  15 years of BCI research at graz university of technology: current projects , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[13]  Horst Bischof,et al.  Toward Self-Paced Brain–Computer Communication: Navigation Through Virtual Worlds , 2008, IEEE Transactions on Biomedical Engineering.

[14]  Horst Bischof,et al.  The Self-Paced Graz Brain-Computer Interface: Methods and Applications , 2007, Comput. Intell. Neurosci..

[15]  Hüseyin Gürüler,et al.  Looking around with your brain in a virtual world , 2011, CCMB.

[16]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[17]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[18]  Russell M. Taylor,et al.  VRPN: a device-independent, network-transparent VR peripheral system , 2001, VRST '01.

[19]  Desney S. Tan,et al.  Brain-Computer Interfacing for Intelligent Systems , 2008, IEEE Intelligent Systems.

[20]  Philippe Coiffet,et al.  Virtual Reality Technology , 2003, Presence: Teleoperators & Virtual Environments.

[21]  A. Clark.  Supersizing the Mind , 2008 .

[22]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[23]  Brendan Z. Allison,et al.  A software SSVEP BCI integrating stimuli within motivating and immersive virtual and augmented reality environments , 2010 .

[24]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[25]  F. Lotte,et al.  Self-Paced Brain-Computer Interaction with Virtual Worlds: A Quantitative and Qualitative Study "Out of the Lab" , 2008 .

[26]  H. R. Hartson,et al.  Advances in Human-Computer Interaction , 1985 .

[27]  Reinhold Scherer,et al.  Towards self-paced ( asynchronous ) Brain-Computer Communication : Navigation through virtual worlds , 2006 .

[28]  Christa Neuper,et al.  Walking by Thinking: The Brainwaves Are Crucial, Not the Muscles! , 2006, PRESENCE: Teleoperators and Virtual Environments.

[29]  Günter Edlinger,et al.  A Hybrid Brain-Computer Interface for Smart Home Control , 2011, HCI.

[30]  E. Donchin,et al.  EEG-based communication: prospects and problems. , 1996, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[31]  Sven Havemann,et al.  DAVE - Eine neue Technologie zur preiswerten und hochqualitativen immersiven 3D-Darstellung , 2003 .

[32]  Heba Lakany,et al.  Human Behavior Integration Improves Classification Rates in Real-Time BCI , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Reinhold Scherer,et al.  Avatar navigation in virtual and augmented reality environments using an SSVEP BCI , 2010 .

[34]  F. L. D. Silva,et al.  Event-Related Desynchronization , 1999 .

[35]  Mel Slater,et al.  Effects of P300-Based BCI Use on Reported Presence in a Virtual Environment , 2010, PRESENCE: Teleoperators and Virtual Environments.

[36]  Fabien Lotte,et al.  Brain-computer interfaces for 3D games: hype or hope? , 2011, FDG.

[37]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[38]  S. Voloshynovskiy,et al.  EEG-Based Synchronized Brain-Computer Interfaces: A Model for Optimizing the Number of Mental Tasks , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  J. Wolpaw,et al.  Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface , 2005, Neurology.

[40]  Touradj Ebrahimi,et al.  Recent advances in brain-computer interfaces , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[41]  J. Wolpaw,et al.  Towards an independent brain–computer interface using steady state visual evoked potentials , 2008, Clinical Neurophysiology.

[42]  Michitaka Hirose,et al.  Brain-Computer Interfaces, Virtual Reality, and Videogames , 2008, Computer.

[43]  G. Pfurtscheller,et al.  Rapid prototyping of an EEG-based brain-computer interface (BCI) , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[44]  Gert Pfurtscheller,et al.  Self-paced exploration of the Austrian National Library through thought , 2007 .

[45]  E. Sellers,et al.  How many people are able to control a P300-based brain–computer interface (BCI)? , 2009, Neuroscience Letters.

[46]  P. Haggard.  Conscious intention and motor cognition , 2005, Trends in Cognitive Sciences.

[47]  John R. Smith,et al.  Steady-State VEP-Based Brain-Computer Interface Control in an Immersive 3D Gaming Environment , 2005, EURASIP J. Adv. Signal Process..

[48]  Robert Leeb.  Brain computer communication: the motivation, aim, and impact of virtual feedback , 2009 .

[49]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[50]  Anatole Lécuyer,et al.  Exploring Large Virtual Environments by Thoughts Using a BrainComputer Interface Based on Motor Imagery and High-Level Commands , 2010, PRESENCE: Teleoperators and Virtual Environments.

[51]  Mel Slater,et al.  Goal-Oriented Control with Brain-Computer Interface , 2009, HCI.

[52]  G Pfurtscheller,et al.  Centrally controlled heart rate changes during mental practice in immersive virtual environment: a case study with a tetraplegic. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[53]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[54]  Anatole Lécuyer,et al.  An overview of research on "passive" brain-computer interfaces for implicit human-computer interaction , 2010 .

[55]  Francisco Velasco-Álvarez,et al.  Free Virtual Navigation Using Motor Imagery Through an Asynchronous BrainComputer Interface , 2010, PRESENCE: Teleoperators and Virtual Environments.

[56]  Janice S. Aikins.  Out of the Lab , 1993, IEEE Expert.

[57]  E Donchin,et al.  The mental prosthesis: assessing the speed of a P300-based brain-computer interface. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[58]  Anton Nijholt,et al.  Turning Shortcomings into Challenges: Brain-Computer Interfaces for Games , 2009, INTETAIN.

[59]  A. Cichocki,et al.  Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives , 2010, Progress in Neurobiology.

[60]  Gert Pfurtscheller,et al.  Walking from thought , 2006, Brain Research.

[61]  Dean J Krusienski,et al.  A comparison of classification techniques for the P300 Speller , 2006, Journal of neural engineering.

[62]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[63]  Mel Slater,et al.  Brain Computer Interface for Virtual Reality Control , 2009, ESANN.

[64]  Francisco Velasco-Álvarez,et al.  A two-class brain computer interface to freely navigate through virtual worlds / Ein Zwei-Klassen-Brain-Computer-Interface zur freien Navigation durch virtuelle Welten , 2009, Biomedizinische Technik. Biomedical engineering.

[65]  J. Pineda,et al.  Learning to control brain rhythms: making a brain-computer interface possible , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[66]  C. Brunia,et al.  Changes in heart rate and slow brain potentials related to motor preparation and stimulus anticipation in a time estimation task. , 1987, Psychophysiology.

[67]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[68]  Gunther Krausz,et al.  ow many people are able to control a P 300-based brain – computer nterface ( BCI ) ? , 2009 .

[69]  Ricardo Ron-Angevin,et al.  Brain–computer interface: Changes in performance using virtual reality techniques , 2009, Neuroscience Letters.

[70]  Christian Jutten,et al.  " Brain Invaders": a prototype of an open-source P300-based video game working with the OpenViBE platform , 2011 .

[71]  Anatole Lécuyer,et al.  Navigating in Virtual Worlds Using a Self-Paced SSVEP-Based Brain–Computer Interface with Integrated Stimulation and Real-Time Feedback , 2011, PRESENCE: Teleoperators and Virtual Environments.

[72]  J. A. Elshout.  Review of Brain-Computer Interfaces based on the P300 evoked potential , 2009 .

[73]  Günter Edlinger,et al.  Hardware/Software Components and Applications of BCIs , 2011 .

[74]  J D Bayliss,et al.  A virtual reality testbed for brain-computer interface research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[75]  M. Jeannerod,et al.  Vegetative response during imagined movement is proportional to mental effort , 1991, Behavioural Brain Research.

[76]  S C Gandevia,et al.  Transcranial magnetic stimulation can influence the selection of motor programmes. , 1990, Journal of neurology, neurosurgery, and psychiatry.

[77]  Benjamin Blankertz,et al.  Designing for uncertain, asymmetric control: Interaction design for brain-computer interfaces , 2009, Int. J. Hum. Comput. Stud..

[78]  Dieter Schmalstieg,et al.  An Application Framework for Controlling an Avatar in a Desktop-Based Virtual Environment via a Software SSVEP BrainComputer Interface , 2010, PRESENCE: Teleoperators and Virtual Environments.

[79]  T. Maeshima,et al.  Autonomic response specificity during motor imagery. , 2000, Journal of physiological anthropology and applied human science.

[80]  Vera Kaiser,et al.  BCI Applications for People with Disabilities: Defining User Needs and User Requirements , 2009 .

[81]  Mel Slater,et al.  Taking steps: the influence of a walking technique on presence in virtual reality , 1995, TCHI.