Inferring Player Experiences Using Facial Expressions Analysis

Understanding player experiences is central to game design. Video captures of players is a common practice for obtaining rich reviewable data for analysing these experiences. However, not enough has been done in investigating ways of preprocessing the video for a more efficient analysis process. This paper consolidates and extends prior work on validating the feasibility of using automated facial expressions analysis as a natural quantitative method for evaluating player experiences. A study was performed on participants playing a first-person puzzle shooter game (Portal 2) and a social drawing trivia game (Draw My Thing), and results were shown to exhibit rich details for inferring player experiences from facial expressions. Significant correlations were also observed between facial expression intensities and self reports from the Game Experience Questionnaire. In particular, the challenge dimension consistently showed positive correlations with anger and joy. This paper eventually presents a case for increasing the application of computer vision in video analyses of gameplay.

[1]  Lennart E. Nacke,et al.  Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment , 2009 .

[2]  Maureen McGranaghan.  A Human Face , 1998 .

[3]  Chek Tien Tan,et al.  A feasibility study in using facial expressions analysis to evaluate player experiences , 2012, IE '12.

[4]  Katherine Isbister,et al.  Game Usability - Advice from the Experts for Advancing the Player Experience , 2008 .

[5]  J. Cohn,et al.  Spontaneous facial expression in a small group can be automatically measured: An initial demonstration , 2010, Behavior research methods.

[6]  Magy Seif El-Nasr,et al.  Visualizing and understanding players' behavior in video games: discovering patterns and supporting aggregation and comparison , 2011, SIGGRAPH 2011.

[7]  Lennart E. Nacke,et al.  Flow and immersion in first-person shooters: measuring the player's gameplay experience , 2008, Future Play.

[8]  Georgios N. Yannakakis,et al.  Correlation between heart rate, electrodermal activity and player experience in first-person shooter games , 2010, Sandbox '10.

[9]  Daniel McDuff,et al.  Real-time inference of mental states from facial expressions and upper body gestures , 2011, Face and Gesture 2011.

[10]  Katherine Isbister,et al.  Game Usability: Advancing the Player Experience , 2008 .

[11]  Magy Seif El-Nasr,et al.  Exploring Quantitative Methods for Evaluating Sports Games , 2010 .

[12]  Nicole Fruehauf.  Flow The Psychology Of Optimal Experience , 2016 .

[13]  Michael John,et al.  Data cracker: developing a visual game analytic tool for analyzing online gameplay , 2011, CHI.

[14]  Enrique Muñoz,et al.  Recognising facial expressions in video sequences , 2007, Pattern Analysis and Applications.

[15]  Chek Tien Tan,et al.  Towards a Non-Disruptive, Practical and Objective Automated Playtesting Process , 2011, IDP@AIIDE.

[16]  Shimon Whiteson,et al.  Towards Personalised Gaming via Facial Expression Recognition , 2014, AIIDE.

[17]  Regan L. Mandryk,et al.  A continuous and objective evaluation of emotional experience with interactive play environments , 2006, CHI.

[18]  M. Csíkszentmihályi.  Flow: The Psychology of Optimal Experience , 1990 .

[19]  Daniel McDuff,et al.  Acume: A new visualization tool for understanding facial expression and gesture data , 2011, Face and Gesture 2011.

[20]  Randy J. Pagulayan,et al.  User-centered design in games , 2012 .

[21]  Mats Liljedahl,et al.  Immersion and Gameplay Experience: A Contingency Framework , 2010, Int. J. Comput. Games Technol..

[22]  J. Russell.  A circumplex model of affect. , 1980 .

[23]  Pedro Branco.  Computer-based facial expression analysis for assessing user experience , 2006 .

[24]  Magy Seif El-Nasr,et al.  A Deeper Look at the Use of Telemetry for Analysis of Player Behavior in RTS Games , 2011, ICEC.

[25]  Lennart E. Nacke,et al.  More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game , 2010, Interact. Comput..

[26]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[27]  Christopher D. Shaw,et al.  Visualizing and understanding players' behavior in video games: discovering patterns and supporting aggregation and comparison , 2011, Sandbox '11.

[28]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .