Capturing user engagement via psychophysiology: measures and mechanisms for biocybernetic adaptation

The concept of task engagement is associated with effortful striving to reach a desired goal. This dimension is fundamental for software designed to elicit high quality performance. This paper will review the concept of task engagement, both in the psychological literature and with respect to affective computing approaches, such as biofeedback and the definition of 'flow' states. This paper will briefly describe a series of laboratory experiments designed to explore measures of task engagement based on EEG and cardiovascular measures. These experiments employed a number of manipulations to influence task engagement, e.g. performance feedback, task difficulty and financial incentives. Results demonstrated the sensitivity of EEG measures to cognitive sources of engagement (e.g. mental workload) whilst cardiovascular variables tended to respond to the motivation to achieve. We use these findings to explore how real-time monitoring of engagement may generate adaptive dynamics for software design using a computer game as an exemplar system.

[1]  Leslie D. Kirby,et al.  Effort determination of cardiovascular response: An integrative analysis with applications in social psychology , 2001 .

[2]  R. Wright,et al.  Refining the Prediction of Effort: Brehm's Distinction between Potential Motivation and Motivation Intensity , 2008 .

[3]  Lennart E. Nacke,et al.  Flow and immersion in first-person shooters: measuring the player's gameplay experience , 2008, Future Play.

[4]  Glenn F. Wilson,et al.  Performance Enhancement in an Uninhabited Air Vehicle Task Using Psychophysiologically Determined Adaptive Aiding , 2007, Hum. Factors.

[5]  F. Freeman,et al.  Evaluation of an adaptive automation system using three EEG indices with a visual tracking task , 1999, Biological Psychology.

[6]  John J. B. Allen,et al.  Frontal EEG asymmetry and the behavioral activation and inhibition systems. , 2003, Psychophysiology.

[7]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[8]  Jenova Chen,et al.  Flow in games (and everything else) , 2007, CACM.

[9]  Rebecca A. Grier,et al.  Fundamental dimensions of subjective state in performance settings: task engagement, distress, and worry. , 2002, Emotion.

[10]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Alan J. Dix,et al.  Using frustration in the design of adaptive videogames , 2004, ACE '04.

[12]  Alan J. Dix,et al.  Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me (ACE) , 2005, DiGRA Conference.

[13]  Jonathan R Wolpaw,et al.  Brain–computer interface systems: progress and prospects , 2007, Expert review of medical devices.

[14]  Michael Richter,et al.  Incentive value, unclear task difficulty, and cardiovascular reactivity in active coping. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[15]  Michael Richter,et al.  Task difficulty effects on cardiac activity. , 2008, Psychophysiology.

[16]  Andrew K. Przybylski,et al.  The Motivational Pull of Video Games: A Self-Determination Theory Approach , 2006 .

[17]  R. Davidson.  What does the prefrontal cortex “do” in affect: perspectives on frontal EEG asymmetry research , 2004, Biological Psychology.

[18]  E. A. Locke,et al.  A theory of goal setting & task performance , 1990 .

[19]  Jonathan Klein,et al.  This computer responds to user frustration: Theory, design, and results , 2002, Interact. Comput..

[20]  J. Beatty.  Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982 .

[21]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[22]  Anthony W. K. Gaillard,et al.  Stress, Workload, and Fatigue as Three Biobehavioral States : A General Overview , 2001 .

[23]  Winslow Burleson,et al.  Affective agents: Sustaining motivation to learn through failure and state of "stuck" , 2004 .

[24]  A. Pope,et al.  Biocybernetic system evaluates indices of operator engagement in automated task , 1995, Biological Psychology.

[25]  G. Mulder.  The Concept and Measurement of Mental Effort , 1986 .

[26]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[27]  Michael E. Smith,et al.  Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods , 1998, Hum. Factors.

[28]  Ashish Kapoor,et al.  Automatic prediction of frustration , 2007, Int. J. Hum. Comput. Stud..

[29]  Guillaume Chanel,et al.  Boredom, engagement and anxiety as indicators for adaptation to difficulty in games , 2008, MindTrek '08.

[30]  Jeffrey B. Henriques,et al.  Frontal Brain Asymmetry and Reward Responsiveness , 2005, Psychological science.

[31]  Anita Miller,et al.  Task-dependent changes in frontal brain asymmetry: effects of incentive cues, outcome expectancies, and motor responses. , 2001, Psychophysiology.

[32]  John J. B. Allen,et al.  Issues and assumptions on the road from raw signals to metrics of frontal EEG asymmetry in emotion , 2004, Biological Psychology.

[33]  Jeff T. Larsen,et al.  Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. , 2003, Psychophysiology.

[34]  Stephen H. Fairclough,et al.  A research agenda for physiological computing , 2004, Interact. Comput..

[35]  Coniferous softwood.  GENERAL TERMS , 2003 .

[36]  Jennifer Allanson.  Electrophysiologically Interactive Computer Systems , 2002, Computer.

[37]  Marko Turpeinen,et al.  Towards Emotionally Adapted Games based on User Controlled Emotion Knobs , 2005, DiGRA Conference.

[38]  Mark W. Scerbo,et al.  A brain-based system for adaptive automation , 2003 .

[39]  John J. B. Allen,et al.  Frontal EEG asymmetry as a moderator and mediator of emotion , 2004, Biological Psychology.

[40]  John J. B. Allen,et al.  Behavioral activation sensitivity and resting frontal EEG asymmetry: covariation of putative indicators related to risk for mood disorders. , 1997, Journal of abnormal psychology.

[41]  Michael G. H. Coles,et al.  Energetical Issues in Research on Human Information Processing , 1986 .