Adaptation in Affective Video Games: A Literature Review

Abstract Playing computer games is a predominantly emotional than rational process. Video games with affect-based adaptation measure player’s behavior signals and recognize player’s emotional states in order to adapt specific game features in a dynamic manner and to improve player’s engagement, immersion, excitement, and challenge. The present review deals with models for the presentation of emotions, techniques for measuring behavioral signals, emotion recognition and adaptation mechanisms applied in video games with affective feedback including methods for their assessment and validation. Studies using self-reports, observational methods and psychophysiological measurements of both autonomic and central nervous systems including processing and interpretation of signals are systematically reviewed with regard to their results concerning emotional adaptation in games. Next, the article provides a comparative analysis of affect-based adaptation methods and techniques applied in 14 adaptive video games developed in last ten years. Based on this comparison, we summarized future directions in research, design and evaluation of video games with affective adaption.

[1]  Eva Cerezo,et al.  Facial emotional classification: from a discrete perspective to a continuous emotional space , 2013, Pattern Analysis and Applications.

[2]  Jennifer Healey,et al.  StartleCam: a cybernetic wearable camera , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[3]  P. Costa,et al.  Validation of the five-factor model of personality across instruments and observers. , 1987, Journal of personality and social psychology.

[4]  Roberto Merletti,et al.  Electromyography. Physiology, engineering and non invasive applications , 2005 .

[5]  Paul J. Diefenbach,et al.  MindTactics: A Brain Computer Interface gaming platform , 2010, 2010 2nd International IEEE Consumer Electronics Society's Games Innovations Conference.

[6]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[7]  A. Dale,et al.  Improved Localizadon of Cortical Activity by Combining EEG and MEG with MRI Cortical Surface Reconstruction: A Linear Approach , 1993, Journal of Cognitive Neuroscience.

[8]  Rosalind W. Picard.  Affective Computing , 1997 .

[9]  Changchun Liu,et al.  Dynamic Difficulty Adjustment in Computer Games Through Real-Time Anxiety-Based Affective Feedback , 2009, Int. J. Hum. Comput. Interact..

[10]  Maria João da Costa Pereira,et al.  Mihaly Csikszentmihalyi. 2002. Fluir: A Psicologia da Experiência Óptima. Título original: Flow: The Psychology of Optimal Experience (1990). Nova Iorque: Harper Perennial. Tradução de Marta Amado. Lisboa: Relógio d’Água Editores , 2002 .

[11]  Guillaume Chanel,et al.  Boredom, engagement and anxiety as indicators for adaptation to difficulty in games , 2008, MindTrek '08.

[12]  Eva Hudlicka,et al.  Affective game engines: motivation and requirements , 2009, FDG.

[13]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[14]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[15]  H. Nagaraja,et al.  Heart rate variability: origins, methods, and interpretive caveats. , 1997, Psychophysiology.

[16]  Brian McDonald,et al.  Intelligent Biofeedback using an Immersive Competitive Environment , 2001 .

[17]  Vasco Pereira Torres,et al.  Development of Biofeedback Mechanisms in a Procedural Environment Using Biometric Sensors , 2013, ArXiv.

[18]  Pete C. Trimmer,et al.  On the Evolution and Optimality of Mood States , 2013, Behavioral sciences.

[19]  M. Coles,et al.  Event-related brain potentials: An introduction. , 1995 .

[20]  Joost Broekens,et al.  AffectButton: A method for reliable and valid affective self-report , 2013, Int. J. Hum. Comput. Stud..

[21]  Thomas S. Huang,et al.  3D facial expression recognition based on automatically selected features , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[22]  Alan J. Dix,et al.  Using frustration in the design of adaptive videogames , 2004, ACE '04.

[23]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[24]  Regan L. Mandryk,et al.  Biofeedback game design: using direct and indirect physiological control to enhance game interaction , 2011, CHI.

[25]  T. Sejnowski,et al.  Measuring facial expressions by computer image analysis. , 1999, Psychophysiology.

[26]  Egon L. van den Broek,et al.  Prerequisites for Affective Signal Processing (ASP) , 2009, BIOSIGNALS.

[27]  Rachael E. Jack.  Culture and facial expressions of emotion , 2013 .

[28]  E. Hudlicka.  AFFECTIVE COMPUTING FOR GAME DESIGN , 2008 .

[29]  Oonagh M. Giggins,et al.  Biofeedback in rehabilitation , 2013, Journal of NeuroEngineering and Rehabilitation.

[30]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[31]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[32]  Erik Champion,et al.  Please Biofeed the Zombies: Enhancing the Gameplay and Display of a Horror Game Using Biofeedback , 2007, DiGRA Conference.

[33]  Richard Rouse.  Games on the verge of a nervous breakdown: emotional content in computer games , 2001, COMG.

[34]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[35]  Tom Alexander Garner,et al.  Game Sound from Behind the Sofa: An Exploration into the Fear Potential of Sound & Psychophysiological Approaches to Audio-centric, Adaptive Gameplay , 2013 .

[36]  H. Jaap van den Herik,et al.  Rapid and Reliable Adaptation of Video Game AI , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[37]  Ana Paiva,et al.  Emotion in Games , 2011, ACII.

[38]  Eugénio C. Oliveira,et al.  Real-Time Psychophysiological Emotional State Estimation in Digital Gameplay Scenarios , 2013, EANN.

[39]  Shimon Whiteson,et al.  Towards Personalised Gaming via Facial Expression Recognition , 2014, AIIDE.

[40]  P. Ekman,et al.  Pan-Cultural Elements in Facial Displays of Emotion , 1969, Science.

[41]  Stephen H. Fairclough,et al.  Construction of the biocybernetic loop: a case study , 2012, ICMI '12.

[42]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[43]  Alan J. Dix,et al.  Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me (ACE) , 2005, DiGRA Conference.

[44]  Luca Chittaro,et al.  Affective computing vs. affective placebo: Study of a biofeedback-controlled game for relaxation training , 2014, Int. J. Hum. Comput. Stud..

[45]  Chek Tien Tan,et al.  A feasibility study in using facial expressions analysis to evaluate player experiences , 2012, IE '12.

[46]  Guillaume Chanel,et al.  A review of the use of psychophysiological methods in game research , 2011 .

[47]  Chek Tien Tan,et al.  Personalised gaming: a motivation and overview of literature , 2012, IE '12.

[48]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[49]  Peta Wyeth,et al.  Revisiting the GameFlow model with detailed heuristics , 2012 .

[50]  Lennart E. Nacke,et al.  Flow and immersion in first-person shooters: measuring the player's gameplay experience , 2008, Future Play.

[51]  Georgios N. Yannakakis,et al.  Real-Time Game Adaptation for Optimizing Player Satisfaction , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[52]  Julian Togelius,et al.  Experience-Driven Procedural Content Generation , 2011, IEEE Trans. Affect. Comput..

[53]  J. Russell,et al.  An approach to environmental psychology , 1974 .

[54]  Hasan Ayaz,et al.  Assessment of Cognitive Neural Correlates for a Functional Near Infrared-Based Brain Computer Interface System , 2009, HCI.

[55]  Kristina Höök,et al.  Affective Loop Experiences - What Are They? , 2008, PERSUASIVE.

[56]  M. Mendl,et al.  An integrative and functional framework for the study of animal emotion and mood , 2010, Proceedings of the Royal Society B: Biological Sciences.

[57]  Timo Partala,et al.  Understanding the most satisfying and unsatisfying user experiences: Emotions, psychological needs, and context , 2012, Interact. Comput..

[58]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[59]  Nilanjan Sarkar,et al.  Maintaining Optimal Challenge in Computer Games through Real-Time Physiological Feedback , 2005 .

[60]  Ludmila I. Kuncheva,et al.  Technological Advancements in Affective Gaming: A Historical Survey , 2014 .

[61]  Reeshad S. Dalal,et al.  Measurement of Emotions , 2012 .

[62]  A. Pope,et al.  Biocybernetic system evaluates indices of operator engagement in automated task , 1995, Biological Psychology.

[63]  Björn W. Schuller,et al.  Emotion representation, analysis and synthesis in continuous space: A survey , 2011, Face and Gesture 2011.

[64]  Amit K. Roy-Chowdhury,et al.  Integrating Motion, Illumination, and Structure in Video Sequences with Applications in Illumination-Invariant Tracking , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Georgios N. Yannakakis,et al.  Entertainment modeling through physiology in physical play , 2008, Int. J. Hum. Comput. Stud..

[66]  Craig A. Lindley,et al.  Game Play Schemas: From Player Analysis to Adaptive Game Mechanics , 2008, Int. J. Comput. Games Technol..

[67]  DichevChristo,et al.  From Gamification to Gameful Design and Gameful Experience in Learning , 2015 .

[68]  Jenova Chen,et al.  Flow in games (and everything else) , 2007, CACM.

[69]  Wijnand A. IJsselsteijn,et al.  Creating an Emotionally Adaptive Game , 2008, ICEC.

[70]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[71]  J. Russell.  A circumplex model of affect. , 1980 .

[72]  Ricardo Gutierrez-Osuna,et al.  A comparative study of game mechanics and control laws for an adaptive physiological game , 2014, Journal on Multimodal User Interfaces.

[73]  M. Csíkszentmihályi.  Finding Flow: The Psychology of Engagement with Everyday Life , 1997 .

[74]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[75]  Chek Tien Tan,et al.  Inferring Player Experiences Using Facial Expressions Analysis , 2014, IE.

[76]  Dylan M. Jones,et al.  Refining the measurement of mood: The UWIST Mood Adjective Checklist , 1990 .

[77]  Clark Verbrugge,et al.  Adaptive companions in FPS games , 2013, FDG.

[78]  Galia Angelova,et al.  Trends and Opportunities in Computer Science OER Development , 2015 .

[79]  Wijnand A. IJsselsteijn,et al.  Dynamic Game Balancing by Recognizing Affect , 2008, Fun and Games.

[80]  Oliver G. B. Garrod,et al.  Facial expressions of emotion are not culturally universal , 2012, Proceedings of the National Academy of Sciences.

[81]  Marko Turpeinen,et al.  The influence of implicit and explicit biofeedback in first-person shooter games , 2010, CHI.

[82]  Dietrich Albert,et al.  Educationally Adaptive: Balancing Serious Games , 2012, Int. J. Comput. Sci. Sport.

[83]  Andrea Bonarini,et al.  Modeling enjoyment preference from physiological responses in a car racing game , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[84]  Jyh-Yeong Chang,et al.  Gaming control using a wearable and wireless EEG-based brain-computer interface device with novel dry foam-based sensors , 2012, Journal of NeuroEngineering and Rehabilitation.

[85]  F. Shaffer,et al.  Heart Rate Variability: New Perspectives on Physiological Mechanisms, Assessment of Self-regulatory Capacity, and Health risk , 2015, Global advances in health and medicine.