Brain–Computer Interfaces Handbook Technological and Theoretical Advances

Play, that is, self-motivated activities for enjoyment, is a significant aspect for human devel-opment and essential to learning and skill acquisition. Games, the structured form of play, are increasingly being used in brain–computer interface (BCI) and neurofeedback (NF) applications. In BCI and NF applications, patterns of the users’ brain activation are assessed in real time and fed back to the users. When users become successful in modulating their own brain activation, improvements in behavior, cognition, or motor function follow or they are able to control external devices such as a computer, wheelchair, or neuroprosthesis. In electroencephalogram-based applications, however, a large number of users cannot attain control over their own brain signals. Current approaches to attaining control require lengthy repetitive trainings. The use of games and game-like feedback aims at keeping user motivation and engagement high over time. This chapter provides an overview of existing game-like feedback modalities and critically discusses their potential value and also possible drawbacks in BCI and NF applications.

[1]  Alexandre N. Tuch,et al.  Towards understanding the effects of individual gamification elements on intrinsic motivation and performance , 2017, Comput. Hum. Behav..

[2]  Christa Neuper,et al.  Effects of a 3D Virtual Reality Neurofeedback Scenario on User Experience and Performance in Stroke Patients , 2016, GALA.

[3]  Gernot R. Müller-Putz,et al.  Brain-computer interface adaptation for an end user to compete in the Cybathlon , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[4]  Gernot R. Müller-Putz,et al.  Lets play Tic-Tac-Toe: A Brain-Computer Interface case study in cerebral palsy , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[5]  David Coyle,et al.  Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy , 2016, JMIR serious games.

[6]  Fabien Lotte,et al.  Why standard brain-computer interface (BCI) training protocols should be changed: an experimental study , 2016, Journal of neural engineering.

[7]  Dimitri Van De Ville,et al.  Meta-analysis of real-time fMRI neurofeedback studies using individual participant data: How is brain regulation mediated? , 2016, NeuroImage.

[8]  C. Neuper,et al.  Specific effects of EEG based neurofeedback training on memory functions in post-stroke victims , 2015, Journal of NeuroEngineering and Rehabilitation.

[9]  C. Neuper,et al.  Resting-state sensorimotor rhythm (SMR) power predicts the ability to up-regulate SMR in an EEG-instrumental conditioning paradigm , 2015, Clinical Neurophysiology.

[10]  Gernot R. Müller-Putz,et al.  Game-Based BCI Training: Interactive Design for Individuals with Cerebral Palsy , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[11]  K. Koschutnig,et al.  Brain volumetry and self-regulation of brain activity relevant for neurofeedback , 2015, Biological Psychology.

[12]  Tele Tan,et al.  3D visualization of movements can amplify motor cortex activation during subsequent motor imagery , 2015, Front. Hum. Neurosci..

[13]  E. Friedrich,et al.  An Effective Neurofeedback Intervention to Improve Social Interactions in Children with Autism Spectrum Disorder , 2015, Journal of Autism and Developmental Disorders.

[14]  Michael Lifshitz,et al.  Neurofeedback, Self-Regulation, and Brain Imaging: Clinical Science and Fad in the Service of Mental Disorders , 2015, Psychotherapy and Psychosomatics.

[15]  Gernot R. Müller-Putz,et al.  Individually Adapted Imagery Improves Brain-Computer Interface Performance in End-Users with Disability , 2015, PloS one.

[16]  M. Molinari,et al.  Brain–computer interface boosts motor imagery practice during stroke recovery , 2015, Annals of neurology.

[17]  L. Yao,et al.  Reorganization of functional brain networks mediates the improvement of cognitive performance following real‐time neurofeedback training of working memory , 2015, Human brain mapping.

[18]  Ana Paiva,et al.  Game elements improve performance in a working memory training task , 2015, Int. J. Serious Games.

[19]  Reinhold Scherer,et al.  Thought-based row-column scanning communication board for individuals with cerebral palsy. , 2015, Annals of physical and rehabilitation medicine.

[20]  Mitchell Harrop,et al.  Paradigms of games research in HCI: a review of 10 years of research at CHI , 2014, CHI PLAY.

[21]  Christa Neuper,et al.  On the need to better specify the concept of “control” in brain-computer-interfaces/neurofeedback research , 2014, Front. Syst. Neurosci..

[22]  X. Li,et al.  The brain structure correlates of individual differences in trait mindfulness: A voxel-based morphometry study , 2014, Neuroscience.

[23]  E. Friedrich,et al.  Brain–computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum , 2014, Front. Neuroeng..

[24]  J. Gruzelier EEG-neurofeedback for optimising performance. I: A review of cognitive and affective outcome in healthy participants , 2014, Neuroscience & Biobehavioral Reviews.

[25]  Giuseppe Sartori,et al.  Attention Training with an Easy-to-Use Brain Computer Interface , 2014, HCI.

[26]  Christa Neuper,et al.  Mind over brain, brain over mind: cognitive causes and consequences of controlling brain activity , 2014, Front. Hum. Neurosci..

[27]  Maud Marchal,et al.  The Mind-Mirror: See your brain in action in your head using EEG and augmented reality , 2014, 2014 IEEE Virtual Reality (VR).

[28]  Sing-Yau Goh,et al.  Effect of mindfulness meditation on brain–computer interface performance , 2014, Consciousness and Cognition.

[29]  C. Neuper,et al.  Neural substrates of cognitive control under the belief of getting neurofeedback training , 2013, Front. Hum. Neurosci..

[30]  C. Neuper,et al.  Learning to modulate one's own brain activity: the effect of spontaneous mental strategies , 2013, Front. Hum. Neurosci..

[31]  C. Neuper,et al.  Whatever Works: A Systematic User-Centered Training Protocol to Optimize Brain-Computer Interfacing Individually , 2013, PloS one.

[32]  Christian Mühl,et al.  Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design , 2013, Front. Hum. Neurosci..

[33]  Christa Neuper,et al.  Control beliefs can predict the ability to up-regulate sensorimotor rhythm during neurofeedback training , 2013, Front. Hum. Neurosci..

[34]  B. Varkuti,et al.  Prediction of brain-computer interface aptitude from individual brain structure , 2013, Front. Hum. Neurosci..

[35]  Anton Nijholt,et al.  Experiencing BCI Control in a Popular Computer Game , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[36]  Gernot R. Müller-Putz,et al.  On the Use of Games for Noninvasive EEG-Based Functional Brain Mapping , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[37]  Brendan Z. Allison,et al.  Brain–computer interfacing: more than the sum of its parts , 2012, Soft Computing.

[38]  Vince D. Calhoun,et al.  Mind over chatter: Plastic up-regulation of the fMRI salience network directly after EEG neurofeedback , 2013, NeuroImage.

[39]  Brian Knutson,et al.  Interactivity and Reward-Related Neural Activation during a Serious Videogame , 2012, PloS one.

[40]  T. Chau,et al.  Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI , 2012, BMC Research Notes.

[41]  K. Müller,et al.  Psychological predictors of SMR-BCI performance , 2012, Biological Psychology.

[42]  Hasan Ayaz,et al.  An optical brain computer interface for environmental control , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[43]  Jonathan Smallwood,et al.  Threatened to distraction Mind-wandering as a consequence of stereotype threat , 2011 .

[44]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[45]  Qiang Wang,et al.  Fractal dimension based neurofeedback in serious games , 2011, The Visual Computer.

[46]  Jee H. Yoo,et al.  Toward a Science of Learning Games , 2011 .

[47]  Stefan Haufe,et al.  The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology , 2010, Front. Neurosci..

[48]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[49]  Milena M. Head,et al.  Computer use by older adults: A multi-disciplinary review , 2010, Comput. Hum. Behav..

[50]  J. Gruzelier,et al.  Acting performance and flow state enhanced with sensory-motor rhythm neurofeedback comparing ecologically valid immersive VR and training screen scenarios , 2010, Neuroscience Letters.

[51]  Klaus-Robert Müller,et al.  Neurophysiological predictor of SMR-based BCI performance , 2010, NeuroImage.

[52]  A. Kübler,et al.  Motivation modulates the P300 amplitude during brain–computer interface use , 2010, Clinical Neurophysiology.

[53]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[54]  Johnell O Brooks,et al.  Simulator sickness during driving simulation studies. , 2010, Accident; analysis and prevention.

[55]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[56]  Sven Haller,et al.  Real-time fMRI feedback training may improve chronic tinnitus , 2010, European Radiology.

[57]  Christa Neuper,et al.  Temporal Coding of Brain Patterns for Direct Limb Control in Humans , 2010, Front. Neurosci..

[58]  J. Keller Motivational Design for Learning and Performance: The ARCS Model Approach , 2009 .

[59]  Maria V. Sanchez-Vives,et al.  How we experience immersive virtual environments: the concept of presence and its measurement * , 2009 .

[60]  M. Arns,et al.  Efficacy of Neurofeedback Treatment in ADHD: The Effects on Inattention, Impulsivity and Hyperactivity: A Meta-Analysis , 2009, Clinical EEG and neuroscience.

[61]  David A. Kaiser,et al.  Meta-Analysis of EEG Biofeedback in Treating Epilepsy , 2009, Clinical EEG and neuroscience.

[62]  Han-Jeong Hwang,et al.  Neurofeedback-based motor imagery training for brain–computer interface (BCI) , 2009, Journal of Neuroscience Methods.

[63]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[64]  Liqing Zhang,et al.  EEG-based asynchronous BCI control of a car in 3D virtual reality environments , 2009 .

[65]  Ricardo Ron-Angevin,et al.  Brain–computer interface: Changes in performance using virtual reality techniques , 2009, Neuroscience Letters.

[66]  Klaus-Robert Müller,et al.  Playing Pinball with non-invasive BCI , 2008, NIPS.

[67]  Horst Bischof,et al.  Toward Self-Paced Brain–Computer Communication: Navigation Through Virtual Worlds , 2008, IEEE Transactions on Biomedical Engineering.

[68]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[69]  G. Pfurtscheller,et al.  Brain–Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[70]  T. Mulder Motor imagery and action observation: cognitive tools for rehabilitation , 2007, Journal of Neural Transmission.

[71]  Gert Pfurtscheller,et al.  Navigating Virtual Reality by Thought: What Is It Like? , 2007, PRESENCE: Teleoperators and Virtual Environments.

[72]  J. Smallwood,et al.  The restless mind. , 2006, Psychological bulletin.

[73]  Klaus-Robert Müller,et al.  Toward noninvasive brain-computer interfaces , 2006, IEEE Signal Process. Mag..

[74]  Christa Neuper,et al.  Walking by Thinking: The Brainwaves Are Crucial, Not the Muscles! , 2006, PRESENCE: Teleoperators and Virtual Environments.

[75]  John D E Gabrieli,et al.  Control over brain activation and pain learned by using real-time functional MRI. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[76]  G. Pfurtscheller,et al.  EEG-based neuroprosthesis control: A step towards clinical practice , 2005, Neuroscience Letters.

[77]  Sun I. Kim,et al.  Neurofeedback Training with Virtual Reality for Inattention and Impulsiveness , 2004, Cyberpsychology Behav. Soc. Netw..

[78]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[79]  Soo-Young Lee,et al.  Brain–computer interface using fMRI: spatial navigation by thoughts , 2004, Neuroreport.

[80]  Gary H. Glover,et al.  Learned regulation of spatially localized brain activation using real-time fMRI , 2004, NeuroImage.

[81]  G. Pfurtscheller,et al.  ‘Thought’ – control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia , 2003, Neuroscience Letters.

[82]  G. Pfurtscheller,et al.  Critical Decision-Speed and Information Transfer in the “Graz Brain–Computer Interface” , 2003, Applied psychophysiology and biofeedback.

[83]  J.D. Bayliss,et al.  Use of the evoked potential P3 component for control in a virtual apartment , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[84]  J D Bayliss,et al.  A virtual reality testbed for brain-computer interface research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[85]  G Calhoun,et al.  Brain-computer interfaces based on the steady-state visual-evoked response. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[86]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[87]  D. Brooks,et al.  Evidence for striatal dopamine release during a video game , 1998, Nature.

[88]  Mel Slater,et al.  Immersion, presence and performance in virtual environments: an experiment with tri-dimensional chess , 1996, VRST.

[89]  Philip H. Mirvis Flow: The Psychology of Optimal Experience , 1991 .

[90]  J. Lacroix,et al.  The acquisition of autonomic control through biofeedback: some tests of discrimination theory. , 1981, Psychophysiology.

[91]  D. Hoang FLOW: The Psychology of Optimal Experience , 2018 .

[92]  Dimitri Van De Ville,et al.  Meta-analysis of real-time fMRI neurofeedback studies: how is brain regulation mediated? , 2018 .

[93]  Gernot R. Müller-Putz,et al.  Games for BCI Skill Learning , 2015 .

[94]  C. Neuper,et al.  Neurofeedback and Serious Games , 2014 .

[95]  Anatole Lécuyer,et al.  Author manuscript, published in "IEEE Transactions on Computational Intelligence and AI in games (2013)" Two Brains, One Game: Design and Evaluation of a Multi-User BCI Video Game Based on Motor Imagery , 2022 .

[96]  Stephan G. Boehm,et al.  Upregulation of emotion areas through neurofeedback with a focus on positive mood , 2011, Cognitive, affective & behavioral neuroscience.

[97]  Brendan Z. Allison,et al.  Could Anyone Use a BCI? , 2010, Brain-Computer Interfaces.

[98]  Bettina Sorger,et al.  Another kind of 'BOLD Response': answering multiple-choice questions via online decoded single-trial brain signals. , 2009, Progress in brain research.

[99]  Reinhold Scherer,et al.  Flexibility and practicality graz brain-computer interface approach. , 2009, International review of neurobiology.

[100]  N. Yan,et al.  Designing a brain-computer interface device for neurofeedback using virtual environments , 2008 .

[101]  Christoph Bartneck,et al.  EEG Headset for Neurofeedback Therapy - Enabling Easy Use in the Home Environment , 2008, BIOSIGNALS.

[102]  S Bermudez i Badia,et al.  A virtual reality system for motor and cognitive neurorehabilitation , 2007 .

[103]  Christa Neuper,et al.  Motor imagery and EEG-based control of spelling devices and neuroprostheses. , 2006, Progress in brain research.

[104]  John R. Smith,et al.  Steady-State VEP-Based Brain-Computer Interface Control in an Immersive 3D Gaming Environment , 2005, EURASIP J. Adv. Signal Process..

[105]  G Pfurtscheller,et al.  Exploring Virtual Environments with an EEG-based BCI through Motor Imagery / Erkundung von virtuellen Welten durch Bewegungsvorstellungen mit Hilfe eines EEG-basierten BCI , 2005, Biomedizinische Technik. Biomedical engineering.

[106]  Francesca Morganti,et al.  Virtual interaction in cognitive neuropsychology , 2004, Cybertherapy.

[107]  N. Birbaumer,et al.  Predictability of Brain-Computer Communication , 2004 .

[108]  J. Lacroix,et al.  Mechanisms of Biofeedback Control , 1986 .

[109]  H. Laborit,et al.  [Experimental study]. , 1958, Bulletin mensuel - Societe de medecine militaire francaise.

[110]  T. Shanmugapriya,et al.  International Journal of Innovative Research in Computer and Communication Engineering Design of Differential Near-infrared Spectroscopy Based Brain Machine Interface , 2022 .