Adapting Cognitive Task Analysis to Elicit the Skill Chain of a Game

Playing a game is a complex skill comprising a set of more basic skills. These skills commonly map onto the main mechanics of the game, and build and depend on each other in a nested learning hierarchy, which game designers have modeled as skill chains made of skill atoms. For players to optimally learn and enjoy a game, it should introduce skill atoms in the ideal sequence of this learning hierarchy. However, game designers typically construct and use hypothetical skill chains based solely on design intent, theory, or personal observation, rather than empirical observation of players. This risks creating incomplete or suboptimal progression designs. In response, this paper presents an adapted cognitive task analysis method for eliciting the empirical skill chain of a game. A case study illustrates and critically reflects the method. While effective in foregrounding overlooked low-level skills required by a game, its efficiency and generalizability remain to be proven.

[1]  Gavriel Salvendy,et al.  The cognitive task analysis methods for job and task design: review and reappraisal , 2004, Behav. Inf. Technol..

[2]  Miguel Sicart,et al.  Defining Game Mechanics , 2008, Game Stud..

[3]  David F. Feldon,et al.  Cognitive Task Analysis–Based Training , 2013 .

[4]  Staffan Björk,et al.  Patterns In Game Design , 2004 .

[5]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[6]  Miguel Nussbaum,et al.  The atomic intrinsic integration approach: A structured methodology for the design of games for the conceptual understanding of physics , 2012, Comput. Educ..

[7]  David Kirschner,et al.  Measuring Video Game Engagement Through Gameplay Reviews , 2014 .

[8]  Herre van Oostendorp,et al.  Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective , 2009 .

[9]  Miguel Sicart,et al.  Loops and Metagames: Understanding Game Design Structures , 2015, FDG.

[10]  Magy Seif El-Nasr,et al.  Methods for Game User Research: Studying Player Behavior to Enhance Game Design , 2013, IEEE Computer Graphics and Applications.

[11]  Ernest Adams,et al.  Game Mechanics: Advanced Game Design , 2012 .

[12]  Nicholas P. Dempsey.  Stimulated Recall Interviews in Ethnography , 2010 .

[13]  Alessandro De Gloria,et al.  Mapping learning and game mechanics for serious games analysis , 2015, Br. J. Educ. Technol..

[14]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[15]  Robin Hunicke,et al.  The case for dynamic difficulty adjustment in games , 2005, ACE '05.

[16]  Anurag Sarkar,et al.  Engagement effects of player rating system-based matchmaking for level ordering in human computation games , 2017, FDG.

[17]  M. Csíkszentmihályi.  Flow: The Psychology of Optimal Experience , 1990 .

[18]  Vincent Aleven,et al.  Using Empirical Learning Curve Analysis to Inform Design in an Educational Game , 2015, CHI PLAY.

[19]  Nicolas Michinov,et al.  Identifying Flow in Video Games: Towards a New Observation-Based Method , 2016, Int. J. Gaming Comput. Mediat. Simulations.

[20]  Anders Drachen,et al.  Methods for Evaluating Gameplay Experience in a Serious Gaming Context , 2010, Int. J. Comput. Sci. Sport.

[21]  M. W. van Someren,et al.  The think aloud method: a practical approach to modelling cognitive processes , 1994 .

[22]  Peter Robinson,et al.  Chapter 1. An introduction to theory and research in task sequencing and instructed second language learning , 2014 .

[23]  Thomas Connolly,et al.  Games-Based Learning Advancements for Multi-Sensory Human Computer Interfaces: Techniques and Effective Practices , 2009 .

[24]  Roberta Calderwood,et al.  Critical decision method for eliciting knowledge , 1989, IEEE Trans. Syst. Man Cybern..

[25]  Robert M. Gagné,et al.  Presidential address of division 15 learning hierarchies , 1968 .

[26]  Wolfgang Hochleitner,et al.  Using Heuristics to Evaluate the Overall User Experience of Video Games and Advanced Interaction Games , 2010, Evaluating User Experience in Games.

[27]  M. Blair,et al.  Video Game Telemetry as a Critical Tool in the Study of Complex Skill Learning , 2013, PloS one.

[28]  J. Dormans,et al.  Engineering emergence: applied theory for game design , 2012 .

[29]  Sebastian Deterding,et al.  The Lens of Intrinsic Skill Atoms: A Method for Gameful Design , 2015, Hum. Comput. Interact..

[30]  D. Hoang.  FLOW: The Psychology of Optimal Experience , 2018 .

[31]  Bonnie L. Tjeerdsma.  A Comparison of Teacher and Student Perspectives of Tasks and Feedback , 1997 .

[32]  Rilla Khaled,et al.  Tales from the front lines of a large-scale serious game project , 2012, CHI.

[33]  M. Csíkszentmihályi.  Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990. , 1990 .

[34]  Ben Kirman,et al.  Learning curves: analysing pace and challenge in four successful puzzle games , 2014, CHI PLAY.

[35]  Regina Bernhaupt.  User Experience Evaluation in Entertainment , 2010, Evaluating User Experience in Games.

[36]  Magy Seif El-Nasr,et al.  Game Analytics , 2013, Springer London.

[37]  Guillaume Chanel,et al.  A review of the use of psychophysiological methods in game research , 2011 .

[38]  Jun Hu,et al.  An activity theory-based model for serious games analysis and conceptual design , 2015, Comput. Educ..

[39]  Thomas L. Seamster,et al.  A Skill-Based Cognitive Task Analysis Framework , 2000 .

[40]  Staffan Björk,et al.  Formal analysis of gameplay , 2015 .

[41]  Flávio S. Corrêa da Silva,et al.  A Systematic Review of Game Design Methods and Tools , 2013, ICEC.

[42]  John Lyle,et al.  Stimulated recall: a report on its use in naturalistic research , 2003 .

[43]  Nancy J. Cooke,et al.  Varieties of knowledge elicitation techniques , 1994, Int. J. Hum. Comput. Stud..

[44]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[45]  David F. Feldon,et al.  Cognitive task analysis , 2009 .

[46]  James Paul Gee,et al.  What video games have to teach us about learning and literacy , 2007, CIE.

[47]  Robert Zubek,et al.  MDA : A Formal Approach to Game Design and Game Research , 2004 .

[48]  Robert M. Gagné,et al.  Past and future research on learning hierarchies 1 , 1974 .

[49]  R. J. Bogumil,et al.  The reflective practitioner: How professionals think in action , 1985, Proceedings of the IEEE.

[50]  Richard E. Clark,et al.  Blueprints for complex learning: The 4C/ID-model , 2002 .

[51]  Gary Klein,et al.  Working Minds: A Practitioner's Guide to Cognitive Task Analysis , 2006 .

[52]  Sumit Gulwani,et al.  Automatic Game Progression Design through Analysis of Solution Features , 2015, CHI.

[53]  Laura Johnson,et al.  How Many Interviews Are Enough? , 2006 .

[54]  Reed Stevens,et al.  In game, In room, In world: Reconnecting video game play to the rest of kids’ lives , 2008 .

[55]  Jessica Hammer,et al.  Playtesting with a Purpose , 2016, CHI PLAY.

[56]  Katherine Isbister,et al.  Chapter 5 – Let the Game Tester Do the Talking: Think Aloud and Interviewing to Learn About the Game Experience , 2008 .

[57]  David H. Jonassen,et al.  Task Analysis Methods for Instructional Design , 1998 .

[58]  Nachiappan Nagappan,et al.  What Drives People: Creating Engagement Profiles of Players from Game Log Data , 2015, CHI PLAY.

[59]  Jori Pitkänen,et al.  Studying thoughts: stimulated recall as a game research method , 2015 .

[60]  David Kirschner,et al.  Experts and Novices or Expertise? Positioning Players through Gameplay Reviews , 2013, DiGRA Conference.

[61]  Charlotte Wiberg,et al.  User Experience Design for Inexperienced Gamers: GAP - Game Approachability Principles , 2010, Evaluating User Experience in Games.

[62]  Alberto J. Cañas,et al.  A TEORIA SUBJACENTE AOS MAPAS CONCEITUAIS E COMO ELABORÁ-LOS E USÁ-LOS * THE THEORY UNDERLYING CONCEPT MAPS AND HOW TO CONSTRUCT AND USE THEM , 2010 .