Bridging Two Worlds: Principled Game-Based Assessment in Industry for Playful Learning at Scale

In recent years, a large body of research in game-based assessment (GBA) has been rooted in the application of methodologies like evidence-centered design to digital game-based learning. This approach affords principled alignment of the target competency with evidence of learning and task design—thus enabling assessment to be embedded in the game experience, allowing student-responsive scaffolding and supporting engagement through formative feedback. Application of these principles to large-scale learning game production can be vital to expanding the benefits of GBA in practice for impact on playful, engaged learning at scale. Practices that support this application to game industry are therefore important, addressing challenges of implementing principled design on short production timelines, sustaining production values to support user adoption and financial sustainability, and integrating with “big data” industry culture to support learning insights (in which disciplines like educational data mining can be particularly relevant). In addressing these challenges, this chapter offers an example of a working GBA practice in an industry context, which implements ECD-based learning design—integrated with principles of educational data mining to inform corresponding event-stream data design—for the production of data-driven educational games to support learning for students at scale. These games can leverage this data-driven approach to support learning in a classroom setting, offering teacher-centered dashboard tools with visualization of student progress and yielding significant learning gains in a recent classroom pilot study with the target learner age group.

[1]  Robert J. Mislevy,et al.  Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining , 2012, EDM 2012.

[2]  Robert J. Mislevy,et al.  An Application of Exploratory Data Analysis in the Development of Game-Based Assessments , 2015 .

[3]  George Siemens,et al.  The Cambridge Handbook of the Learning Sciences: Educational Data Mining and Learning Analytics , 2014 .

[4]  Jonathan M. Kevan,et al.  Experience API: Flexible, Decentralized and Activity-Centric Data Collection , 2015, Technology, Knowledge and Learning.

[5]  Jesper Juul The Art of Failure: An Essay on the Pain of Playing Video Games , 2013 .

[6]  V. Elizabeth Owen,et al.  Game-based assessment: an integrated model for capturing evidence of learning in play , 2014, Int. J. Learn. Technol..

[7]  Kurt Squire,et al.  Adage: an open API for data collection in educational games , 2014, CHI PLAY.

[8]  Jody Clarke-Midura,et al.  Better Learning in Games: A Balanced Design Lens for a New Generation of Learning Games , 2015 .

[9]  Douglas B. Clark,et al.  Investigating Epistemic Stances in Game Play with Data Mining , 2017, Int. J. Gaming Comput. Mediat. Simulations.

[10]  Lindsay D. Grace A linguistic analysis of games: Verbs and nouns for content estimation , 2014, FDG.

[11]  Ryan Shaun Joazeiro de Baker,et al.  A Comparison of Face-based and Interaction-based Affect Detectors in Physics Playground , 2015, EDM.

[12]  K. Squire From Content to Context: Videogames as Designed Experience , 2006 .

[13]  Manu Kapur,et al.  Playing Epistemic Games in Science and Mathematics Classrooms. , 2010 .

[14]  James Paul Gee,et al.  What video games have to teach us about learning and literacy , 2007, CIE.

[15]  Julie Sarama,et al.  Building Blocks for early childhood mathematics , 2003 .

[16]  Jody Clarke,et al.  Predicting Successful Inquiry Learning in a Virtual Performance Assessment for Science , 2013, UMAP.

[17]  Shauna J. Sweet,et al.  Using the ECD Framework to Support Evidentiary Reasoning in the Context of a Simulation Study for Detecting Learner Differences in Epistemic Games , 2012, EDM 2012.

[18]  Gordon Erlebacher,et al.  Designing Dynamic Support for Game-Based Learning , 2018, Interdisciplinary Design of Game-based Learning Platforms.

[19]  Robert J. Mislevy,et al.  Evidence-Centered Design for Simulation-Based Assessment. CRESST Report 800. , 2011 .

[20]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[21]  Magy Seif El-Nasr,et al.  In Your Face(t) Impact of Personality and Context on Gameplay Behavior , 2015, FDG.

[22]  Katie Salen,et al.  Rules of play: game design fundamentals , 2003 .

[23]  Andy Johnson,et al.  Applying standards to systematize learning analytics in serious games , 2017, Comput. Stand. Interfaces.

[24]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[25]  J. Sarama,et al.  Learning Trajectories in Mathematics Education , 2004 .

[26]  Samuel Greiff,et al.  Measuring problem solving skills via stealth assessment in an engaging video game , 2016, Comput. Hum. Behav..

[27]  Kristen E. DiCerbo,et al.  Detecting Player Goals from Game Log Files , 2013, EDM.

[28]  Ryan Shaun Joazeiro de Baker,et al.  Modeling User Exploration and Boundary Testing in Digital Learning Games , 2016, UMAP.

[29]  Espen Aarseth,et al.  I Fought the Law: Transgressive Play and The Implied Player , 2007, DiGRA Conference.

[30]  Jesse Schell,et al.  The Art of Game Design: A book of lenses , 2019 .

[31]  R. Almond,et al.  A BRIEF INTRODUCTION TO EVIDENCE-CENTERED DESIGN , 2003 .

[32]  Yue Gong,et al.  Wheel-Spinning: Students Who Fail to Master a Skill , 2013, AIED.

[33]  Ryan S. Baker,et al.  Educational Data Mining and Learning Analytics , 2014 .

[34]  Jodi Asbell-Clarke,et al.  Using game analytics to evaluate puzzle design and level progression in a serious game , 2016, LAK.

[35]  Yoon Jeon Kim,et al.  Formative and Stealth Assessment , 2014 .

[36]  Dan Yngve Jacobsen,et al.  Dropping Out or Dropping In? A Connectivist Approach to Understanding Participants’ Strategies in an e-Learning MOOC Pilot , 2019, Technol. Knowl. Learn..

[37]  Neil T. Heffernan,et al.  Decision Tree Modeling of Wheel-Spinning and Productive Persistence in Skill Builders. , 2018 .

[38]  Lloyd P. Rieber,et al.  Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games , 1996 .

[39]  Gregory K. W. K. Chung,et al.  Identifying Key Features of Student Performance in Educational Video Games and Simulations through Cluster Analysis , 2012, EDM 2012.

[40]  Ken Perlin,et al.  Metrics in Simulations and Games for Learning , 2013, Game Analytics, Maximizing the Value of Player Data.

[41]  Ryan Shaun Joazeiro de Baker,et al.  Modeling how incoming knowledge, persistence, affective states, and in-game progress influence student learning from an educational game , 2015, Comput. Educ..

[42]  Robert J. Mislevy,et al.  Taming Log Files From Game/Simulation‐Based Assessments: Data Models and Data Analysis Tools , 2016 .

[43]  V. Elizabeth Owen,et al.  Fueling Prediction of Player Decisions: Foundations of Feature Engineering for Optimized Behavior Modeling in Serious Games , 2018, Technol. Knowl. Learn..

[44]  S. Stefan,et al.  A Typology of Players in the Game Physics Playground , 2017 .

[45]  James Paul Gee,et al.  Learning by Design: Good Video Games as Learning Machines , 2005 .

[46]  Sean C. Duncan,et al.  Scientific Habits of Mind in Virtual Worlds , 2008 .

[47]  Gregory K. W. K. Chung Guidelines for the Design and Implementation of Game Telemetry for Serious Games Analytics , 2015 .

[48]  Kurt Squire,et al.  Video Games and Learning: Teaching and Participatory Culture in the Digital Age , 2011 .

[49]  Kristen E. DiCerbo Task Models in the Digital Ocean , 2014 .

[50]  Jodi Asbell-Clarke,et al.  Assessment design for emergent game-based learning , 2013, CHI Extended Abstracts.

[51]  Dirk Ifenthaler,et al.  Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives , 2012 .

[52]  Clara Fernández-Vara,et al.  Towards an Ontological Language for Game Analysis , 2005, DiGRA Conference.