Evaluation of an Adaptive Game that Uses EEG Measures Validated during the Design Process as Inputs to a Biocybernetic Loop

Biocybernetic adaptation is a form of physiological computing whereby real-time data streaming from the brain and body is used by a negative control loop to adapt the user interface. This article describes the development of an adaptive game system that is designed to maximize player engagement by utilizing changes in real-time electroencephalography (EEG) to adjust the level of game demand. The research consists of four main stages: (1) the development of a conceptual framework upon which to model the interaction between person and system; (2) the validation of the psychophysiological inference underpinning the loop; (3) the construction of a working prototype; and (4) an evaluation of the adaptive game. Two studies are reported. The first demonstrates the sensitivity of EEG power in the (frontal) theta and (parietal) alpha bands to changing levels of game demand. These variables were then reformulated within the working biocybernetic control loop designed to maximize player engagement. The second study evaluated the performance of an adaptive game of Tetris with respect to system behavior and user experience. Important issues for the design and evaluation of closed-loop interfaces are discussed.

[1]  R. Wright,et al.  Refining the Prediction of Effort: Brehm's Distinction between Potential Motivation and Motivation Intensity , 2008 .

[2]  W. Klimesch,et al.  EEG alpha synchronization and functional coupling during top‐down processing in a working memory task , 2005, Human brain mapping.

[3]  Carolyne R Swain,et al.  Electrophysiological, behavioral, and subjective indexes of workload when performing multiple tasks: manipulations of task difficulty and training. , 1999, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  Mark W. Scerbo,et al.  Comparison of a Brain-Based Adaptive System and a Manual Adaptable System for Invoking Automation , 2006, Hum. Factors.

[5]  Ashish Kapoor,et al.  Automatic prediction of frustration , 2007, Int. J. Hum. Comput. Stud..

[6]  Carryl L. Baldwin,et al.  Adaptive training using an artificial neural network and EEG metrics for within- and cross-task workload classification , 2012, NeuroImage.

[7]  Glenn F. Wilson,et al.  An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures , 2002 .

[8]  Charan Ranganath,et al.  Frontal midline theta oscillations during working memory maintenance and episodic encoding and retrieval , 2014, NeuroImage.

[9]  N. Yeung,et al.  The roles of cortical oscillations in sustained attention , 2015, Trends in Cognitive Sciences.

[10]  Alan J. Dix,et al.  Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me (ACE) , 2005, DiGRA Conference.

[11]  Michael Richter,et al.  Task difficulty effects on cardiac activity. , 2008, Psychophysiology.

[12]  J. Cacioppo,et al.  Inferring psychological significance from physiological signals. , 1990, The American psychologist.

[13]  M. Coltheart.  Hemispheric asymmetry , 1978, Nature.

[14]  O. Jensen,et al.  Frontal theta activity in humans increases with memory load in a working memory task , 2002, The European journal of neuroscience.

[15]  F. Freeman,et al.  Evaluation of an adaptive automation system using three EEG indices with a visual tracking task , 1999, Biological Psychology.

[16]  Raja Parasuraman,et al.  Three Experiments Examining the Use of Electroencephalogram, Event-Related Potentials, and Heart-Rate Variability for Real- Time Human-Centered Adaptive Automation Design , 2003 .

[17]  Regan L. Mandryk,et al.  Biofeedback game design: using direct and indirect physiological control to enhance game interaction , 2011, CHI.

[18]  A. Yoshino,et al.  A topographic electrophysiologic study of mental rotation. , 2000, Brain research. Cognitive brain research.

[19]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[20]  Dylan M. Jones,et al.  Refining the measurement of mood: The UWIST Mood Adjective Checklist , 1990 .

[21]  J. C. Dill,et al.  Blood pressure responses and incentive appraisals as a function of perceived ability and objective task demand. , 1993, Psychophysiology.

[22]  Coniferous softwood.  GENERAL TERMS , 2003 .

[23]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[24]  James C. Christensen,et al.  The effects of day-to-day variability of physiological data on operator functional state classification , 2012, NeuroImage.

[25]  G. Pfurtscheller.  Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest. , 1992, Electroencephalography and clinical neurophysiology.

[26]  Erik Champion,et al.  Please Biofeed the Zombies: Enhancing the Gameplay and Display of a Horror Game Using Biofeedback , 2007, DiGRA Conference.

[27]  A. Lleras,et al.  Brief and rare mental “breaks” keep you focused: Deactivation and reactivation of task goals preempt vigilance decrements , 2010, Cognition.

[28]  M. Frank,et al.  Frontal theta as a mechanism for cognitive control , 2014, Trends in Cognitive Sciences.

[29]  D. Hoang.  FLOW: The Psychology of Optimal Experience , 2018 .

[30]  A. Pope,et al.  Biocybernetic system evaluates indices of operator engagement in automated task , 1995, Biological Psychology.

[31]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[32]  Mark W. Scerbo,et al.  A brain-based system for adaptive automation , 2003 .

[33]  Stephen H. Fairclough,et al.  Capturing user engagement via psychophysiology: measures and mechanisms for biocybernetic adaptation , 2013, Int. J. Auton. Adapt. Commun. Syst..

[34]  R. Piferi,et al.  An alternative approach for achieving cardiovascular baseline: viewing an aquatic video. , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[35]  F. Freeman,et al.  A Closed-Loop System for Examining Psychophysiological Measures for Adaptive Task Allocation , 2000, The International journal of aviation psychology.

[36]  W. Klimesch.  EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[37]  Leslie D. Kirby,et al.  Effort determination of cardiovascular response: An integrative analysis with applications in social psychology , 2001 .

[38]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[39]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[40]  Guillaume Chanel,et al.  Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[41]  Guillaume Chanel,et al.  Boredom, engagement and anxiety as indicators for adaptation to difficulty in games , 2008, MindTrek '08.

[42]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[43]  Changchun Liu,et al.  Affect Recognition in Robot Assisted Rehabilitation of Children with Autism Spectrum Disorder , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[44]  Soichiro Nomura,et al.  Topographic study of human event-related potentials using a task requiring mental rotation , 1998, Neuroscience Letters.

[45]  Lennart E. Nacke,et al.  Flow and immersion in first-person shooters: measuring the player's gameplay experience , 2008, Future Play.

[46]  Michael E. Smith,et al.  Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods , 1998, Hum. Factors.

[47]  Michael Richter,et al.  The heart contracts to reward: monetary incentives and preejection period. , 2009, Psychophysiology.

[48]  Christian Mühl,et al.  Connecting Brains and Bodies: Applying Physiological Computing to Support Social Interaction , 2015, Interact. Comput..

[49]  Jenova Chen,et al.  Flow in games (and everything else) , 2007, CACM.

[50]  Cynthia D. Fisher,et al.  The effects of personal control, competence, and extrinsic reward systems on intrinsic motivation , 1978 .

[51]  Andrew K. Przybylski,et al.  The Motivational Pull of Video Games: A Self-Determination Theory Approach , 2006 .