Modeling cognitive load and physiological arousal through pupil diameter and heart rate

This study investigates individuals’ cognitive load processing abilities while engaged on a decision-making task in serious games, to explore how a substantial cognitive load dominates over the physiological arousal effect on pupil diameter. A serious game was presented to the participants, which displayed the on–line biofeedback based on physiological measurements of arousal. In such dynamic decision-making environment, the pupil diameter was analyzed in relation to the heart rate, to evaluate if the former could be a useful measure of cognitive abilities of individuals. As pupil might reflect both cognitive activity and physiological arousal, the pupillary response will show an arousal effect only when the cognitive demands of the situation are minimal. Evidence shows that in a situation where a substantial level of cognitive activity is required, only that activity will be observable on the pupil diameter, dominating over the physiological arousal effect indicated by the pupillary response. It is suggested that it might be possible to design serious games tailored to the cognitive abilities of an individual player, using the proposed physiological measurements to observe the moment when such dominance occurs.

[1]  S. Knardahl.  Cardiovascular psychophysiology , 2000, Annals of medicine.

[2]  C. Sennersten,et al.  Dissecting Play - Investigating the Cognitive and Emotional Motivations and Affects of Computer Gameplay 1 , 2008 .

[3]  Darryl Charles,et al.  Dynamic Player Modelling: A Framework for Player-Centered Digital Games , 2004 .

[4]  Paul Verhaeghen,et al.  The ties to unbind: age-related differences in feature (un)binding in working memory for emotional faces , 2014, Front. Psychol..

[5]  Karolien Poels,et al.  Playability and player experience research , 2009 .

[6]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[7]  E. Granholm,et al.  Pupillometric measures of cognitive and emotional processes. , 2004, International Journal of Psychophysiology.

[8]  Dawei Wang,et al.  The effect of emotion and time pressure on risk decision-making , 2015 .

[9]  Chung-Ho Su,et al.  The effects of students' motivation, cognitive load and learning anxiety in gamification software engineering education: a structural equation modeling study , 2015, Multimedia Tools and Applications.

[10]  Stephen H. Boutcher,et al.  Cardiovascular response to Stroop: Effect of verbal response and task difficulty , 2006, Biological Psychology.

[11]  Pejman Mirza-Babaei,et al.  Improving Gameplay with Game Metrics and Player Metrics , 2013, Game Analytics, Maximizing the Value of Player Data.

[12]  S. Johnstone,et al.  Varying task difficulty in the Go/Nogo task: the effects of inhibitory control, arousal, and perceived effort on ERP components. , 2013, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  G. Berntson,et al.  Handbook of Psychophysiology: Cardiovascular Psychophysiology , 2007 .

[14]  Wanhui Wen,et al.  Analysis of affective ECG signals toward emotion recognition , 2010 .

[15]  Yi-Hung Huang,et al.  Augmenting teacher-student interaction in digital learning through affective computing , 2017, Multimedia Tools and Applications.

[16]  S. Porges.  The polyvagal perspective , 2007, Biological Psychology.

[17]  Olle Hilborn,et al.  A Serious Game using Physiological Interfaces for Emotion regulation Training in the Context of Financial Decision-Making , 2012, ECIS.

[18]  J. Russell,et al.  The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology , 2005, Development and Psychopathology.

[19]  Gilbert Babin,et al.  Business simulation training in information technology education: Guidelines for new approaches in IT training , 2011 .

[20]  Nienke Meulman,et al.  An ERP study on L2 syntax processing: When do learners fail? , 2014, Front. Psychol..

[21]  M. A. Recarte,et al.  Mental workload while driving: effects on visual search, discrimination, and decision making. , 2003, Journal of experimental psychology. Applied.

[22]  Nilanjan Sarkar,et al.  Anxiety-based affective communication for implicit human–machine interaction , 2022 .

[23]  S. Filipović,et al.  Psychophysiology: Human Behavior and Physiological Response (4th ed.) , 2001 .

[24]  Veikko Surakka,et al.  Pupil size variation as an indication of affective processing , 2003, Int. J. Hum. Comput. Stud..

[25]  S. Sirois,et al.  Pupillometry , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[26]  M. Bradley,et al.  Emotion and motivation I: defensive and appetitive reactions in picture processing. , 2001, Emotion.

[27]  M. Bradley,et al.  Looking at pictures: affective, facial, visceral, and behavioral reactions. , 1993, Psychophysiology.

[28]  P. Hamilton,et al.  Open source ECG analysis , 2002, Computers in Cardiology.

[29]  Gari D. Clifford,et al.  ECG Statistics , Noise , Artifacts , and Missing Data , 2006 .

[30]  Tytus Sosnowski,et al.  Program running versus problem solving: mental task effect on tonic heart rate. , 2004, Psychophysiology.

[31]  J. Jolles,et al.  Pupil dilation in response preparation. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[32]  Christof Weinhardt,et al.  Integrating Biosignals into Information Systems: A NeuroIS Tool for Improving Emotion Regulation , 2013, J. Manag. Inf. Syst..

[33]  Craig A. Lindley,et al.  An Investigation of Visual Attention in FPS Computer Gameplay , 2009, 2009 Conference in Games and Virtual Worlds for Serious Applications.

[34]  S. Steinhauer,et al.  Sympathetic and parasympathetic innervation of pupillary dilation during sustained processing. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[35]  M. Hitchens,et al.  Group play: determining factors on the gaming experience in multiplayer role-playing games , 2008, CIE.

[36]  Robert F. Stanners,et al.  The pupillary response as an indicator of arousal and cognition , 1979 .

[37]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[38]  O. Lowenstein,et al.  Pupillary Reflex Shapes and Topical Clinical Diagnosis , 1955, Neurology.

[39]  Darryl Charles,et al.  Player-Centred Game Design: Player Modelling and Adaptive Digital Games , 2005 .

[40]  M. Bradley,et al.  The pupil as a measure of emotional arousal and autonomic activation. , 2008, Psychophysiology.

[41]  Abdulmotaleb El-Saddik,et al.  U-biofeedback: a multimedia-based reference model for ubiquitous biofeedback systems , 2013, Multimedia Tools and Applications.

[42]  Marius B. Müller,et al.  Auction Fever! How Time Pressure and Social Competition Affect Bidders’ Arousal and Bids in Retail Auctions , 2015 .

[43]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[44]  J. Andreassi.  Psychophysiology: Human Behavior and Physiological Response , 1980 .

[45]  Pietro Guardini,et al.  Better Game Experience Through Game Metrics: A Rally Videogame Case Study , 2013, Game Analytics, Maximizing the Value of Player Data.

[46]  Tarja Susi,et al.  Serious Games : An Overview , 2007 .

[47]  L. Curtin,et al.  The Yerkes-Dodson law. , 1984, Nursing management.