Comparing usability and engagement between a serious game and a traditional online program

A frequent motivation behind the development of serious games is the notion that game play provides a greater level of motivation or engagement that encourages players to perform better or complete more difficult challenges. There are in truth many aspects to concepts such as engagement, immersion, flow or other terms used to motivate the use of games in place of more traditional didactic training approaches. In this paper we discuss some of the issues of usability and its relation to engagement. This discussion is framed in relation to a usability study of a serious game that is being developed to support psychological counseling of individuals with comorbidity of depression and alcohol use disorders. We compare issues of usability and engagement in the serious game called Shadow directly with a more traditional online intervention program called SHADE. The study uses previously established surveys, namely, the System Usability Scale, the Perceived Usefulness and Ease of Use, and a modified survey for measuring Engagement to compare the two approaches. Results from a pilot study with 10 participants frame a discussion of the role usability plays in designing for engagement in serious games such as Shadow.

[1]  Mariano Alcañiz Raya,et al.  Learning in serious virtual worlds: Evaluation of learning effectiveness and appeal to students in the E-Junior project , 2010, Comput. Educ..

[2]  Joel M. Hektner,et al.  Experience sampling method , 2007 .

[3]  Shawn Y. Holmes,et al.  Assessing twenty‐first century skills through a teacher created video game for high school biology students , 2010 .

[4]  Jeanne B. Funk,et al.  A Biopsychosocial Analysis of the Video Game-playing Experience of Children and Adults in the United States , 2006 .

[5]  N. Newton,et al.  A systematic review of computerised serious educational games about alcohol and other drugs for adolescents. , 2014, Drug and alcohol review.

[6]  James C. Lester,et al.  Affect and Engagement in Game-BasedLearning Environments , 2014, IEEE Transactions on Affective Computing.

[7]  Andrew Hargadon,et al.  The pleasures of immersion and engagement: schemas, scripts and the fifth business , 2001, Digit. Creativity.

[8]  A. Elliot,et al.  A 2 X 2 achievement goal framework. , 2001, Journal of personality and social psychology.

[9]  Paul A. Cairns,et al.  Not doing but thinking: the role of challenge in the gaming experience , 2012, CHI.

[10]  Connie Malamed,et al.  Book Review: 'The Gamification of Learning and Instruction: Game-Based Methods and Strategies For Training And Education' by Karl Kapp , 2012, ELERN.

[11]  Darryl Charles,et al.  Player-Centred Game Design : Player Modelling and Adaptive Digital Games , 2005 .

[12]  Suzanne M. Roche,et al.  Absorption: Nature, assessment, and correlates. , 1990 .

[13]  David Salesin,et al.  The challenge of designing scientific discovery games , 2010, FDG.

[14]  D. DavisFred,et al.  User Acceptance of Computer Technology , 1989 .

[15]  Ernest Adams,et al.  Fundamentals of Game Design , 2006 .

[16]  H. Simon,et al.  How to Study Thinking in Everyday Life: Contrasting Think-Aloud Protocols With Descriptions and Explanations of Thinking , 1998 .

[17]  James Minogue,et al.  Investigating the impact of video games on high school students' engagement and learning about genetics , 2009, Comput. Educ..

[18]  Kurt Squire,et al.  Video Game–Based Learning: An Emerging Paradigm for Instruction , 2008 .

[19]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[20]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[21]  Thomas Bjørner,et al.  'Sure, I Would Like to Continue' A Method for Mapping the Experience of Engagement in Video Games , 2012 .

[22]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[23]  Hiroshi Ishii,et al.  Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information , 1998, CoBuild.

[24]  Nadia Garnefski,et al.  Cognitive emotion regulation questionnaire - development of a short 18-item version (CERQ-short). , 2006 .

[25]  Christoph Klimmt,et al.  A Process Model of the Formation of Spatial Presence Experiences , 2007 .

[26]  Rosa María Baños,et al.  Immersion and Emotion: Their Impact on the Sense of Presence , 2004, Cyberpsychology Behav. Soc. Netw..

[27]  Keith Nesbitt,et al.  Training for Life: Designing a Game to Engage Younger People in a Psychological Counselling Program , 2013, Aust. J. Intell. Inf. Process. Syst..

[28]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[29]  J. Glicksohn,et al.  Explorations in Virtual Reality: Absorption, Cognition and Altered State of Consciousness , 1997 .

[30]  Nadia Bianchi-Berthouze,et al.  Stirring up experience through movement in game play: effects on engagement and social behaviour , 2008, CHI.

[31]  Christopher J Patrick,et al.  Psychopathy, startle blink modulation, and electrodermal reactivity in twin men. , 2005, Psychophysiology.

[32]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[33]  Alf Inge Wang,et al.  An Evaluation of a Mobile Game Concept for Lectures , 2008, 2008 21st Conference on Software Engineering Education and Training.

[34]  John W. Rice,et al.  The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[35]  M. Fardo.  KAPP, Karl M. The gamification of learning and instruction: game-based methods and strategies for training and education. San Francisco: Pfeiffer, 2012. , 2013 .

[36]  Jonathan P. Rowe,et al.  Integrating Learning, Problem Solving, and Engagement in Narrative-Centered Learning Environments , 2011, Int. J. Artif. Intell. Educ..

[37]  D. Hoang.  FLOW: The Psychology of Optimal Experience , 2018 .

[38]  Karolien Poels,et al.  Characterising and measuring user experiences in digital games , 2007 .

[39]  Jane McGonigal.  'This Is Not a Game': Immersive Aesthetics and Collective Play , 2003 .

[40]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[41]  H. Irwin,et al.  Pathological and nonpathological dissociation: the relevance of childhood trauma. , 1999, The Journal of psychology.

[42]  Arthur C. Graesser,et al.  What Makes Learning Fun? Exploring the Influence of Choice and Difficulty on Mind Wandering and Engagement during Learning , 2013, AIED.

[43]  Peter A. Todd,et al.  Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication , 1992, MIS Q..

[44]  Mohammad Chuttur,et al.  Overview of the Technology Acceptance Model: Origins, Developments and Future Directions , 2009 .

[45]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[46]  Timothy Paul Cronan,et al.  On the Test-Retest Reliability of Perceived Usefulness and Perceived Ease of Use Scales , 1993 .

[47]  D. Norman.  The psychology of everyday things", Basic Books Inc , 1988 .

[48]  Mark Stansfield,et al.  Evaluation of a game to teach requirements collection and analysis in software engineering at tertiary education level , 2011, Comput. Educ..

[49]  Mark Weiser,et al.  Designing Calm Technology , 2004 .

[50]  W. G. Smith,et al.  Angry Birds , 1888, Nature.

[51]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[52]  Janice Baranowski,et al.  Squire's Quest! Dietary outcome evaluation of a multimedia game. , 2003, American journal of preventive medicine.

[53]  Geertje Bekebrede,et al.  Reviewing the need for gaming in education to accommodate the net generation , 2011, Comput. Educ..

[54]  Thomas W. Malone,et al.  Heuristics for designing enjoyable user interfaces: Lessons from computer games , 1982, CHI '82.

[55]  Nicoletta Adamo-Villani,et al.  Using a Serious Game Approach to Teach 'Operator Precedence' to Introductory Programming Students , 2013, 2013 17th International Conference on Information Visualisation.

[56]  M. Csíkszentmihályi.  Creativity: Flow and the Psychology of Discovery and Invention , 1996 .

[57]  S. Greenberg,et al.  The Psychology of Everyday Things , 2012 .

[58]  Amanda L Baker,et al.  Clinician‐assisted computerised versus therapist‐delivered treatment for depressive and addictive disorders: a randomised controlled trial , 2011, The Medical journal of Australia.

[59]  John L. Sherry,et al.  Sex Differences in Video Game Play: , 2004, Commun. Res..

[60]  M. Csíkszentmihályi,et al.  Experience Sampling Method: Measuring the Quality of Everyday Life , 2006 .

[61]  Karen Blackmore,et al.  Using the Startle Eye-Blink to Measure Affect in Players , 2015 .

[62]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[63]  Brianno Coller,et al.  Video Game-Based Education in Mechanical Engineering: A Look at Student Engagement* , 2009 .

[64]  Katerina Mania,et al.  The Effects of Levels of Immersion on Memory and Presence in Virtual Environments: A Reality Centered Approach , 2001, Cyberpsychology Behav. Soc. Netw..

[65]  Paul T. Costa,et al.  Personality in Adulthood: A Five-Factor Theory Perspective , 2005 .

[66]  B. Cuthbert,et al.  Committee report: Guidelines for human startle eyeblink electromyographic studies. , 2005, Psychophysiology.

[67]  Richard P. Bagozzi,et al.  The Legacy of the Technology Acceptance Model and a Proposal for a Paradigm Shift , 2007, J. Assoc. Inf. Syst..

[68]  Vernon Pizer,et al.  The United States Army , 1967 .

[69]  Marianne Webb,et al.  Reach Out Central: a serious game designed to engage young men to improve mental health and wellbeing , 2010, The Medical journal of Australia.

[70]  Albert H. Segars,et al.  Re-examining perceived ease of use and usefulness , 1993 .

[71]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[72]  D. Simons,et al.  The effects of video game playing on attention, memory, and executive control. , 2008, Acta psychologica.

[73]  Miriam Vollenbroek-Hutten,et al.  Chronic pain rehabilitation with a serious game using multimodal input , 2011, 2011 International Conference on Virtual Rehabilitation.

[74]  Girish H. Subramanian,et al.  A Replication of Perceived Usefulness and Perceived Ease of Use Measurement , 1994 .

[75]  Deborah I Thompson,et al.  Playing for real: video games and stories for health-related behavior change. , 2008, American journal of preventive medicine.

[76]  Dara Meldrum,et al.  Virtual reality rehabilitation of balance: assessment of the usability of the Nintendo Wii® Fit Plus , 2012, Disability and rehabilitation. Assistive technology.

[77]  Christine L. Lisetti,et al.  A User-Modeling Approach to Build User's Psycho-Physiological Maps of Emotions using Bio-Sensors , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[78]  Charlotte Wiberg,et al.  Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration , 2009, HCI.

[79]  Jaak Panksepp,et al.  Neuroimaging Helps to Clarify Brain Affective Processing Without Necessarily Clarifying Emotions , 2013 .