Heuristic usability evaluation on games: a modular approach

Heuristic evaluation is the preferred method to assess usability in games when experts conduct this evaluation. Many heuristics guidelines have been proposed attending to specificities of games but they only focus on specific subsets of games or platforms. In fact, to date the most used guideline to evaluate games usability is still Nielsen’s proposal, which is focused on generic software. As a result, most evaluations do not cover important aspects in games such as mobility, multiplayer interactions, enjoyability and playability, etc. To promote the usage of new heuristics adapted to different game and platform aspects we propose a modular approach based on the classification of existing game heuristics using metadata and a tool, MUSE (Meta-heUristics uSability Evaluation tool) for games, which allows a rebuild of heuristic guidelines based on metadata selection in order to obtain a customized list for every real evaluation case. The usage of these new rebuilt heuristic guidelines allows an explicit attendance to a wide range of usability aspects in games and a better detection of usability issues. We preliminarily evaluate MUSE with an analysis of two different games, using both the Nielsen’s heuristics and the customized heuristic lists generated by our tool.

[1]  Thomas W. Malone,et al.  Heuristics for designing enjoyable user interfaces: Lessons from computer games , 1982, CHI '82.

[2]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[3]  Wolmet Barendregt,et al.  Identifying usability and fun problems in a computer game during first use and after some practice , 2006, Int. J. Hum. Comput. Stud..

[4]  R. Caillois,et al.  Man, Play and Games , 1958 .

[5]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[6]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[7]  Morten Hertzum,et al.  The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[8]  Aki Järvinen,et al.  Communication and Community in Digital Entertainment Services. Prestudy Research Report , 2002 .

[9]  Wolfgang Hochleitner,et al.  Using Heuristics to Evaluate the Overall User Experience of Video Games and Advanced Interaction Games , 2010, Evaluating User Experience in Games.

[10]  David Myers The Nature of Computer Games: Play As Semiosis (Digital Formations;, V. 16,) , 2003 .

[11]  Richard L. Hazlett,et al.  Measuring emotional valence during interactive experiences: boys at video game play , 2006, CHI.

[12]  Francisco Luis Gutiérrez Vela,et al.  Playability: analysing user experience in video games , 2012, Behav. Inf. Technol..

[13]  Carsten Röcker,et al.  Exploring the Usability of Video Game Heuristics for Pervasive Game Development in Smart Home Environments , 2006 .

[14]  Paul A. Cairns,et al.  Behaviour, realism and immersion in games , 2005, CHI Extended Abstracts.

[15]  Hannu Korhonen,et al.  The explanatory power of playability heuristics , 2011, Advances in Computer Entertainment Technology.

[16]  David Pinelle,et al.  Heuristic evaluation for games: usability principles for video game design , 2008, CHI.

[17]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[18]  Jakob Nielsen,et al.  Chapter 4 – The Usability Engineering Lifecycle , 1993 .

[19]  Virpi Roto,et al.  Towards a shared definition of user experience , 2008, CHI Extended Abstracts.

[20]  Erik Frekjmr,et al.  Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated? , 2000 .

[21]  Lennart E. Nacke,et al.  From playability to a hierarchical game usability model , 2009, Future Play.

[22]  R. Bernhaupt,et al.  Characterising and measuring user experiences in digital games , 2007 .

[23]  Elina M. I. Koivisto,et al.  Playability heuristics for mobile games , 2006, Mobile HCI.

[24]  T. Krzywinska,et al.  Screenplay: Cinema/Videogames/Interfaces , 2002 .

[25]  Mark J.P. Wolf,et al.  Medium of the Video Game , 2002 .

[26]  Chuck Clanton,et al.  An interpreted demonstration of computer game design , 1998, CHI Conference Summary.

[27]  Thomas W. Malone,et al.  Toward a Theory of Intrinsically Motivating Instruction , 1981, Cogn. Sci..

[28]  Franca Garzotto,et al.  Investigating the educational effectiveness of multiplayer online games for children , 2007, IDC.

[29]  Regan L. Mandryk,et al.  Critic-proofing: how using critic reviews and game genres can refine heuristic evaluations , 2010, Future Play.

[30]  Michalis Nik Xenos,et al.  Usability Assessment Heuristics in New Genre Videogames , 2009, 2009 13th Panhellenic Conference on Informatics.

[31]  Paul S. Breidenbach,et al.  The Medium of the Video Game , 2005 .

[32]  Charlotte Wiberg,et al.  Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration , 2009, HCI.

[33]  Carl Gutwin,et al.  Usability heuristics for networked multiplayer games , 2009, GROUP.

[34]  Rosa Yáñez Gómez,et al.  Heuristic Evaluation on Mobile Interfaces: A New Checklist , 2014, TheScientificWorldJournal.

[35]  José Luis Sevillano,et al.  Academic methods for usability evaluation of serious games: a systematic review , 2017, Multimedia Tools and Applications.

[36]  Elina M. I. Koivisto,et al.  Playability heuristics for mobile multi-player games , 2007, DIMEA.

[37]  Gavriel Salvendy,et al.  Number of people required for usability evaluation , 2010, Commun. ACM.

[38]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[39]  Daniel M. Johnson,et al.  Effective affective user interface design in games , 2003, Ergonomics.

[40]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[41]  Marc Hassenzahl,et al.  User experience (UX): towards an experiential perspective on product quality , 2008, IHM '08.

[42]  Darryl Charles,et al.  Toward an understanding of flow in video games , 2008, CIE.

[43]  Harry Hochheiser,et al.  Research Methods for Human-Computer Interaction , 2008 .

[44]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[45]  Tom Apperley Genre and game studies: Toward a critical approach to video game genres , 2006 .

[46]  Janne Paavilainen,et al.  Expert review method in game evaluations: comparison of two playability heuristic sets , 2009, MindTrek '09.

[47]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[48]  Melissa A. Federoff,et al.  HEURISTICS AND USABILITY GUIDELINES FOR THE CREATION AND EVALUATION OF FUN IN VIDEO GAMES , 2002 .

[49]  M. Csíkszentmihályi Play and Intrinsic Rewards , 2014 .

[50]  Tarja Susi,et al.  Serious Games : An Overview , 2007 .

[51]  Heather Desurvire,et al.  Using heuristics to evaluate the playability of games , 2004, CHI EA '04.

[52]  Hannu Korhonen,et al.  Comparison of playtesting and expert review methods in mobile game evaluation , 2010, Fun and Games '10.

[53]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[54]  Fengfeng Ke,et al.  Alternative goal structures for computer game-based learning , 2008, Int. J. Comput. Support. Collab. Learn..