Serious Games Usability Testing: How to Ensure Proper Usability, Playability, and Effectiveness

Usability testing is an important, yet often overlooked, aspect of serious game development. Issues in usability can drastically impact user experience and thus the learning outcomes associated with serious games. The goal of this paper is to provide serious game developers with an approach to efficiently and effectively apply usability testing into their development process. We propose a three-tiered approach to the assessment of game usability with the addition of assessments playability and learning to traditional usability. Learning or training is the main objective of a serious game and enjoyment is often required when trying to elicit the necessary usage to achieve this goal. Step-by-step procedures and associated measures are provided to assess usability, playability, and learning outcomes concurrently with game development, while taking into account the unique goals and limitations of time, personnel, and budget that small development companies often encounter.

[1]  Katherine Isbister,et al.  Game Usability - Advice from the Experts for Advancing the Player Experience , 2008 .

[2]  D. Dillman.  Mail and telephone surveys : the total design method , 1979 .

[3]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[4]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[5]  S. Joy Mountford,et al.  The Art of Human-Computer Interface Design , 1990 .

[6]  Hy Resnick,et al.  Computerized Games in the Human Services—An Introduction , 2008 .

[7]  Ben Shneiderman,et al.  Designing The User Interface , 2013 .

[8]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[9]  Timothy Cleary,et al.  Business Information Technology , 1998 .

[10]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[11]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[12]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[13]  Mun Y. Yi,et al.  Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model , 2003, Int. J. Hum. Comput. Stud..

[14]  Hy Resnick.  Electronic Tools for Social Work Practice and Education , 1994 .

[15]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[16]  J. Conley.  Asking questions: A practical guide to questionnaire design , 1983 .

[17]  R. Eklund,et al.  The Flow Scales Manual , 2004 .

[18]  Katherine Isbister,et al.  Chapter 5 – Let the Game Tester Do the Talking: Think Aloud and Interviewing to Learn About the Game Experience , 2008 .

[19]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .