Methods for Evaluating Gameplay Experience in a Serious Gaming Context

Gameplay experience (GX) is created during the process of player-game interaction, where this interaction has the goal to provide a motivating, fun experience for the player. Since GX is an important factor for the success of failure of a game, a formal classification of how to design for and evaluate GX is necessary. Using appropriate mechanisms for evaluation and measurement of GX allows the validation of good gameplay experiences. This paper presents an approach to formalize such evaluative methods and a roadmap for applying these mechanisms in the context of serious games. We first discuss related work of user experience (UX) and player experience models, based on which we propose a three-layer framework of GX. For each layer, a number of measurement methodologies are listed and our focus is put on physiological and technical metrics for game evaluation. Finally, we point out the potential use of this framework within the field of game-based learning and serious gaming for sports and health. GAMEPLAY, AFFECT, PSYCHOPHYSIOLOGY, USER EXPERIENCE (UX), SERIOUS GAMING

[1]  Thomas W. Malone,et al.  What makes things fun to learn? heuristics for designing instructional computer games , 1980, SIGSMALL '80.

[2]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[3]  J. Jacko,et al.  The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications , 2002 .

[4]  J. Russell.  A circumplex model of affect. , 1980 .

[5]  Amyris Fernandez.  Fun Experience with Digital Games : a Model Proposition , 2007 .

[6]  Leena Arhippainen,et al.  Empirical Evaluation of User Experience in two Adaptive Mobile Application Prototypes , 2003 .

[7]  Miguel Nussbaum,et al.  Playability in Action Videogames: A Qualitative Design Model , 2002, Hum. Comput. Interact..

[8]  Jesse James Garrett.  The Elements of User Experience: User-Centered Design for the Web , 2002 .

[9]  Elina M. I. Koivisto,et al.  Playability heuristics for mobile games , 2006, Mobile HCI.

[10]  Bruce Phillips,et al.  Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems , 2008, CHI.

[11]  Regina Bruder,et al.  Qualität von computergestützten Lernumgebungen beurteilen : ein ganzheitliches Konzept , 2005 .

[12]  Darryl Charles,et al.  Toward an understanding of flow in video games , 2008, CIE.

[13]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[14]  Heather Desurvire,et al.  Using heuristics to evaluate the playability of games , 2004, CHI EA '04.

[15]  Deborah I Thompson,et al.  Playing for real: video games and stories for health-related behavior change. , 2008, American journal of preventive medicine.

[16]  Francisco Luis Gutiérrez Vela,et al.  From Usability to Playability: Introduction to Player-Centred Video Game Development Process , 2009, HCI.

[17]  Alan J. Dix,et al.  Using frustration in the design of adaptive videogames , 2004, ACE '04.

[18]  Lennart E. Nacke,et al.  Log Who's Playing: Psychophysiological Game Analysis Made Easy through Event Logging , 2008, Fun and Games.

[19]  Vincent P. Wade,et al.  Adaptive Educational Games: Providing Non-invasive Personalised Learning Experiences , 2008, 2008 Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning.

[20]  Aki Järvinen,et al.  Communication and Community in Digital Entertainment Services. Prestudy Research Report , 2002 .

[21]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[22]  W. Green,et al.  Pleasure with Products: Human Factors for Body, Mind and Soul , 1999 .

[23]  Regan L. Mandryk,et al.  A continuous and objective evaluation of emotional experience with interactive play environments , 2006, CHI.

[24]  James Paul Gee,et al.  What video games have to teach us about learning and literacy , 2007, CIE.

[25]  Dennis R. Wixon,et al.  Using the RITE method to improve products; a definition and a case study , 2007 .

[26]  Effie Lai-Chong Law,et al.  Towards a UX manifesto , 2007 .

[27]  Stefan Göbel,et al.  80Days: Adaptive Digital Storytelling for Digital Educational Games , 2009, STEG@ICWL.

[28]  Robert J. Logan,et al.  Behavioral and emotional usability: Thomson Consumer Electronics , 1994 .

[29]  Marc Prensky,et al.  Digital game-based learning , 2000, CIE.

[30]  Yvonne de Kort,et al.  Shared Fun Is Doubled Fun: Player Enjoyment as a Function of Social Setting , 2008, Fun and Games.

[31]  Lennart E. Nacke,et al.  Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment , 2009 .

[32]  Georgios N. Yannakakis,et al.  Player modeling using self-organization in Tomb Raider: Underworld , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[33]  Alessandro Canossa,et al.  Defining personas in games using metrics , 2008, Future Play.

[34]  F. Mäyrä,et al.  Fundamental Components of the Gameplay Experience , 2011 .

[35]  David R. Michael,et al.  Serious Games: Games That Educate, Train, and Inform , 2005 .

[36]  Frans Mäyrä,et al.  Fundamental Components of the Gameplay Experience: Analysing Immersion , 2005, DiGRA Conference.

[37]  Randy J. Pagulayan,et al.  User-centered design in games , 2012 .

[38]  W. A. Zellmer.  Toward understanding. , 1988, American journal of hospital pharmacy.

[39]  Christoph Klimmt,et al.  A Process Model of the Formation of Spatial Presence Experiences , 2007 .

[40]  Marc Hassenzahl,et al.  MARC HASSENZAHL CHAPTER 3 The Thing and I: Understanding the Relationship Between User and Product , 2003 .

[41]  Mel Slater,et al.  Presence and The Sixth Sense , 2002, Presence: Teleoperators & Virtual Environments.

[42]  Michael Wiklund,et al.  Usability in practice: how companies develop user-friendly products , 1994 .

[43]  Katherine Isbister,et al.  Game Usability - Advice from the Experts for Advancing the Player Experience , 2008 .

[44]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[45]  Anu Kankainen,et al.  UCPCD: user-centered product concept design , 2003, DUX '03.

[46]  Lennart E. Nacke,et al.  From playability to a hierarchical game usability model , 2009, Future Play.

[47]  Raimund Dachselt,et al.  Advanced gaze visualizations for three-dimensional virtual environments , 2010, ETRA.