A review of the use of psychophysiological methods in game research

This article reviews the psychophysiological method in game research. Psychophysiological measurements provide an objective, continuous, real-time, noninvasive, precise and sensitive way to assess the game experience. However, the best results require controlled experiments with careful monitoring of variables, large enough sample sizes and expertise in electrical signal processing. We briefly explain the theory behind the method and present the most useful measures: electromyography (EMG), electrodermal activity (EDA), electroencephalography (EEG) and cardiac measures. We review previous studies that have used psychophysiological measures in game research and illustrate some future directions. Our article covers several research lines using the psychophysiological method in game studies, and offers a comprehensive list of references for those interested in the field.

[1]  P. Lang.  The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[2]  Charlotte Sennersten.  Gameplay (3D Game Engine + Ray Tracing = Visual Attention through Eye Tracking) , 2008 .

[3]  Ron Tamborini,et al.  What Do We Really Know About First-Person-Shooter Games? An Event-Related, High-Resolution Content Analysis , 2009, J. Comput. Mediat. Commun..

[4]  Georgios N. Yannakakis,et al.  Entertainment modeling through physiology in physical play , 2008, Int. J. Hum. Comput. Stud..

[5]  Guillaume Chanel,et al.  Social interaction using mobile devices and biofeedback: effects on presence, attraction and emotions , 2010 .

[6]  Georgios N. Yannakakis,et al.  Towards affective camera control in games , 2010, User Modeling and User-Adapted Interaction.

[7]  P. Ekman,et al.  Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique. , 1994, Psychological bulletin.

[8]  Katie Salen,et al.  Rules of play: game design fundamentals , 2003 .

[9]  Nagisa Munekata,et al.  A computer game using galvanic skin response , 2003, International Conference on Evolutionary Computation.

[10]  John J. B. Allen,et al.  Manipulation of frontal EEG asymmetry through biofeedback alters self-reported emotional responses and facial EMG. , 2001, Psychophysiology.

[11]  Wijnand A. IJsselsteijn,et al.  Dynamic Game Balancing by Recognizing Affect , 2008, Fun and Games.

[12]  Jussi Holopainen,et al.  The Psychophysiology of Video Gaming: Phasic Emotional Responses to Game Events , 2005, DiGRA Conference.

[13]  Richard L. Hazlett,et al.  Measuring emotional valence during interactive experiences: boys at video game play , 2006, CHI.

[14]  Michele J. Fleming,et al.  Effects of Violent Versus Nonviolent Video Games on Children's Arousal, Aggressive Mood, and Positive Mood , 2001 .

[15]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[16]  M. Ballard,et al.  Mortal Kombat (tm): The Effects of Violent Videogame Play on Males' Hostility and Cardiovascular Responding , 1996 .

[17]  J. Russell,et al.  The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology , 2005, Development and Psychopathology.

[18]  N. Rickard.  Intense emotional responses to music: a test of the physiological arousal hypothesis , 2004 .

[19]  S. Hébert,et al.  Physiological stress response to video-game playing: the contribution of built-in music. , 2005, Life sciences.

[20]  W. Waterink,et al.  Facial and jaw-elevator EMG activity in relation to changes in performance level during a sustained information processing task , 1994, Biological Psychology.

[21]  W. Prinz,et al.  Movements, actions and tool-use actions: an ideomotor approach to imitation , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[22]  Lennart E. Nacke,et al.  Affective Ludology, Flow and Immersion in a First- Person Shooter: Measurement of Player Experience , 2009, ArXiv.

[23]  Erik Champion,et al.  Please Biofeed the Zombies: Enhancing the Gameplay and Display of a Horror Game Using Biofeedback , 2007, DiGRA Conference.

[24]  L. Aftanas,et al.  Analysis of Evoked EEG Synchronization and Desynchronization in Conditions of Emotional Activation in Humans: Temporal and Topographic Characteristics , 2004, Neuroscience and Behavioral Physiology.

[25]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[26]  Yvonne de Kort,et al.  Shared Fun Is Doubled Fun: Player Enjoyment as a Function of Social Setting , 2008, Fun and Games.

[27]  Lennart E. Nacke,et al.  Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment , 2009 .

[28]  Alan T. Pope,et al.  Helping Video Games Rewire "Our Minds" , 2001 .

[29]  Byron Reeves,et al.  Being in the Game: Effects of Avatar Choice and Point of View on Psychophysiological Responses During Play , 2009 .

[30]  Changchun Liu,et al.  An empirical study of machine learning techniques for affect recognition in human–robot interaction , 2006, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  J. Vaidya,et al.  The two general activation systems of affect: Structural findings, evolutionary considerations, and psychobiological evidence , 1999 .

[32]  Yvonne de Kort,et al.  Effects of Sensory Immersion on Behavioural Indicators of Player Experience: Movement Synchrony and Controller Pressure , 2009, DiGRA Conference.

[33]  S. Puttonen,et al.  The psychophysiology of James Bond: phasic emotional responses to violent video game events. , 2008, Emotion.

[34]  Marko Turpeinen,et al.  The influence of implicit and explicit biofeedback in first-person shooter games , 2010, CHI.

[35]  Guillaume Chanel,et al.  Boredom, engagement and anxiety as indicators for adaptation to difficulty in games , 2008, MindTrek '08.

[36]  Robert G. Magee,et al.  You Can't Take It with You? Effects of Handheld Portable Media Consoles on Physiological and Psychological Responses to Video Game and Movie Content , 2009, Cyberpsychology Behav. Soc. Netw..

[37]  Paul D. Bolls,et al.  The Effects of Message Valence and Listener Arousal on Attention, Memory, and Facial Muscular Responses to Radio Advertisements , 2001, Commun. Res..

[38]  Christopher P. Barlett,et al.  Video Game Effects—Confirmed, Suspected, and Speculative , 2009 .

[39]  S. Grüsser,et al.  Specific cue reactivity on computer game-related cues in excessive gamers. , 2007, Behavioral neuroscience.

[40]  Changchun Liu,et al.  Dynamic Difficulty Adjustment in Computer Games Through Real-Time Anxiety-Based Affective Feedback , 2009, Int. J. Hum. Comput. Interact..

[41]  I. McGilchrist,et al.  The Master and His Emissary: The Divided Brain and the Making of the Western World , 2009 .

[42]  K. Isbister,et al.  Chapter 14 – Physiological Measures for Game Evaluation* , 2008 .

[43]  S. Makeig,et al.  Imaging human EEG dynamics using independent component analysis , 2006, Neuroscience & Biobehavioral Reviews.

[44]  Kai Vogeley,et al.  The effects of self-involvement on attention, arousal, and facial expression during social interaction with virtual others: A psychophysiological study , 2006, Social neuroscience.

[45]  R. Lazarus.  Emotion and Adaptation , 1991 .

[46]  D. Watson,et al.  On the Dimensional and Hierarchical Structure of Affect , 1999 .

[47]  Richard J. Davidson,et al.  Validation of ICA-based myogenic artifact correction for scalp and source-localized EEG , 2010, NeuroImage.

[48]  René Weber,et al.  Think Aloud during fMRI: Neuronal Correlates of Subjective Experience in Video Games , 2008, Fun and Games.

[49]  Simon Brown,et al.  Affective gaming: measuring emotion through the gamepad , 2003, CHI Extended Abstracts.

[50]  Perttu Hämäläinen,et al.  Using heart rate to control an interactive game , 2007, CHI.

[51]  K. Kallinen,et al.  Presence and emotion in computer game players during 1st person vs. 3rd person playing view: evidence from self-report, eye-tracking, and facial muscle activity data , 2007 .

[52]  C. Ferguson.  Evidence for publication bias in video game violence effects literature: A meta-analytic review , 2007 .

[53]  Samuel D. Bradley,et al.  Death with a story: How story impacts emotional, motivational, and physiological responses to first-person shooter video games , 2004 .

[54]  Robert F. Potter,et al.  Smile and the Virtual World Smiles With You: Electromyographic Response to Avatar Facial Expressions , 2009 .

[55]  Alexander Toet,et al.  Is a Dark Virtual Environment Scary? , 2009, Cyberpsychology Behav. Soc. Netw..

[56]  Regan L. Mandryk,et al.  Physiological indicators for the evaluation of co-located collaborative play , 2004, CSCW.

[57]  Lennart E. Nacke,et al.  Electroencephalographic Assessment of Player Experience , 2011 .

[58]  Stephen H. Fairclough,et al.  Capturing user engagement via psychophysiology: measures and mechanisms for biocybernetic adaptation , 2013, Int. J. Auton. Adapt. Commun. Syst..

[59]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[60]  M. Benedek,et al.  Decomposition of skin conductance data by means of nonnegative deconvolution , 2010, Psychophysiology.

[61]  Antti Poikola,et al.  Invisible eni: using gaze and pupil size to control a game , 2008, CHI Extended Abstracts.

[62]  Christopher P. Barlett,et al.  The effect of the amount of blood in a violent video game on aggression, hostility, and arousal , 2008 .

[63]  Marta Kutas,et al.  Identifying reliable independent components via split-half comparisons , 2009, NeuroImage.

[64]  N. Ravaja.  Contributions of Psychophysiology to Media Research: Review and Recommendations , 2004 .

[65]  Carson Jonathan Reynolds,et al.  The sensing and measurement of frustration with computers , 2001 .

[66]  Guillaume Chanel,et al.  Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[67]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[68]  Guillaume Chanel,et al.  Psychophysiological methods for game experience research: directions for assessing social experience , 2010 .

[69]  M. Dawson,et al.  The electrodermal system , 2007 .

[70]  C. Ferguson.  The Good, The Bad and the Ugly: A Meta-analytic Review of Positive and Negative Effects of Violent Video Games , 2007, Psychiatric Quarterly.

[71]  K. Kallinen,et al.  The Role of Mood in the Processing of Media Messages From a Small Screen: Effects on Subjective and Physiological Responses , 2006 .

[72]  Simone Mulargia,et al.  The Video Game Theory Reader 2 , 2011 .

[73]  K. P. Kallio,et al.  At Least Nine Ways to Play: Approaching Gamer Mentalities , 2011, Games Cult..

[74]  M. Bradley,et al.  Looking at pictures: affective, facial, visceral, and behavioral reactions. , 1993, Psychophysiology.

[75]  Byron Reeves,et al.  Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player , 2010, Int. J. Hum. Comput. Stud..

[76]  Artur Lugmayr,et al.  Proceedings of the 12th international conference on Entertainment and media in the ubiquitous era , 2008 .

[77]  L. McEvoy,et al.  Neurophysiological indices of strategy development and skill acquisition. , 1999, Brain research. Cognitive brain research.

[78]  R. Tafalla.  Gender Differences in Cardiovascular Reactivity and Game Performance Related to Sensory Modality in Violent Video Game Play1 , 2007 .

[79]  Paul Pauli,et al.  Electromyographic responses to static and dynamic avatar emotional facial expressions. , 2006, Psychophysiology.

[80]  J. Russell.  A circumplex model of affect. , 1980 .

[81]  Sara Ilstedt Hjelm,et al.  Research + design: the making of Brainball , 2003, INTR.

[82]  S. Wolfson,et al.  The effects of sound and colour on responses to a computer game , 2000, Interact. Comput..

[83]  Steve Strand,et al.  Discovering statistics using SPSS, 2nd edition , 2006 .

[84]  James Paul Gee,et al.  What video games have to teach us about learning and literacy , 2007, CIE.

[85]  Aki Järvinen.  Understanding Video Games as Emotional Experiences , 2008 .

[86]  Alan J. Dix,et al.  Using frustration in the design of adaptive videogames , 2004, ACE '04.

[87]  N. Ravaja.  The Psychophysiology of Digital Gaming: The Effect of a Non Co-Located Opponent , 2009 .

[88]  B. Parkinson,et al.  Emotion and motivation , 1995 .

[89]  Frithjof Staude-Müller,et al.  Hostile and Hardened? An Experimental Study on (De-)Sensitization to Violence and Suffering Through Playing Video Games , 2008 .

[90]  L. Walker,et al.  Applied social psychology , 2000 .

[91]  J. Cacioppo,et al.  The skeletomotor system: Surface electromyography. , 2007 .

[92]  Darryl Charles,et al.  Analyzing player behavior in Pacman using feature-driven decision theoretic predictive modeling , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[93]  Qiang Wang,et al.  Fractal dimension based neurofeedback in serious games , 2011, The Visual Computer.

[94]  Michael Zyda,et al.  Introducing Emotion into Military Simulation and Video Game Design America's Army: Operations and VIRTE , 2002, GAME-ON.

[95]  Georgios N. Yannakakis,et al.  Player modeling using self-organization in Tomb Raider: Underworld , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[96]  R. Larsen,et al.  Promises and problems with the circumplex model of emotion. , 1992 .

[97]  Niklas Ravaja,et al.  Oscillatory Brain Responses Evoked by Video Game Events: The Case of Super Monkey Ball 2 , 2007, Cyberpsychology Behav. Soc. Netw..

[98]  R. Davidson.  Affective Style and Affective Disorders: Perspectives from Affective Neuroscience , 1998 .

[99]  K. Kallinen,et al.  Phasic Emotional Reactions to Video Game Events: A Psychophysiological Investigation , 2006 .

[100]  Lennart E. Nacke,et al.  More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game , 2010, Interact. Comput..

[101]  Brian McDonald,et al.  Intelligent Biofeedback using an Immersive Competitive Environment , 2001 .

[102]  Marko Turpeinen,et al.  Spatial Presence and Emotions during Video Game Playing: Does It Matter with Whom You Play? , 2006, PRESENCE: Teleoperators and Virtual Environments.