Learning Analytics as an Assessment Tool in Serious Games: A Review of Literature

The purpose of this chapter is to conduct a systematic review of research on studies using analytics (particularly in-game data such as logs) in serious games (SG) to understand what research has been conducted and what research evidences there are in using analytics in SG to support teaching and learning. The findings of this review showed learner performance and game design strategies were the two most common researched topics. Other topics included motivation and engagement, student behavior, problem solving, learner progress trajectories, and student collaboration. In addressing students’ learning performance, more studies reported that SG had a positive impact on learning; and many highlighted the importance of game design. Some of the studies reviewed also indicated the challenges for researchers to use in-game dynamic data as a research measure. Several trends are identified and implications for future research are discussed.

[1]  Calvin C. Y. Liao,et al.  My-Mini-Pet: a handheld pet-nurturing game to engage students in arithmetic practices , 2011, J. Comput. Assist. Learn..

[2]  Miguel Ángel Conde González,et al.  Visualisation of student learning model in serious games , 2015, Comput. Hum. Behav..

[3]  Hsiao-Ching She,et al.  Learning through playing Virtual Age: Exploring the interactions among student concept learning, gaming performance, in-game behaviors, and the use of in-game characters , 2015, Comput. Educ..

[4]  Gregory K. W. K. Chung,et al.  Identifying Key Features of Student Performance in Educational Video Games and Simulations through Cluster Analysis , 2012, EDM 2012.

[5]  C. Hart Doing a literature review: releasing the social science research imagination. , 1998 .

[6]  Laura K. Allen,et al.  Does agency matter?: Exploring the impact of controlled behaviors within a game-based environment , 2015, Comput. Educ..

[7]  Andrea Gauthier,et al.  Exploring the influence of game design on learning and voluntary use in an online vascular anatomy study aid , 2015, Comput. Educ..

[8]  Chuen-Tsai Sun,et al.  How digital scaffolds in games direct problem-solving behaviors , 2011, Comput. Educ..

[9]  Jaejin Lee,et al.  Examining Through Visualization What Tools Learners Access as They Play a Serious Game for Middle School Science , 2015 .

[10]  Jonathan P. Rowe,et al.  Problem Solving and Game-Based Learning: Effects of Middle Grade Students' Hypothesis Testing Strategies on Learning Outcomes , 2011 .

[11]  Chiara Francalanci,et al.  DashMash: A Mashup Environment for End User Development , 2011, ICWE.

[12]  Christian Sebastian Loh,et al.  Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics , 2013, Education and Information Technologies.

[13]  Raija Hämäläinen,et al.  Designing and evaluating collaboration in a virtual game environment for vocational learning , 2008, Comput. Educ..

[14]  R. Kozma,et al.  Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena , 1997 .

[15]  Christian Sebastian Loh,et al.  Predicting expert-novice performance as serious games analytics with objective-oriented and navigational action sequences , 2015, Comput. Hum. Behav..

[16]  Deirdre Kerr Using Data Mining Results to Improve Educational Video Game Design , 2015, EDM 2015.

[17]  Min Liu,et al.  A Look at Research on Mobile Learning in K–12 Education From 2007 to the Present , 2014 .

[18]  Andrew Harrison,et al.  Timed Report Measures Learning: Game-Based Embedded Assessment , 2012 .

[19]  Huei-Tse Hou,et al.  Exploring the behavioral patterns of learners in an educational massively multiple online role-playing game (MMORPG) , 2012, Comput. Educ..

[20]  Yuan-Bang Cheng,et al.  The effect of simulation games on the learning of computational problem solving , 2011, Comput. Educ..

[21]  Vivian Hsueh-hua Chen,et al.  Ubiquitous games for learning (UbiqGames): Weatherlings, a worked example , 2012, J. Comput. Assist. Learn..

[22]  M. Csíkszentmihályi Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990. , 1990 .

[23]  Christian Sebastian Loh,et al.  Maximum Similarity Index (MSI): A metric to differentiate the performance of novices vs. multiple-experts in serious games , 2014, Comput. Hum. Behav..

[24]  Debbie Denise Reese,et al.  Video game learning dynamics: Actionable measures of multidimensional learning trajectories , 2015, Br. J. Educ. Technol..

[25]  Arthur C. Graesser,et al.  Operation ARIES!: Methods, Mystery, and Mixed Models: Discourse Features Predict Affect in a Serious Game , 2013, EDM 2013.

[26]  K. Kiili Content creation challenges and flow experience in educational games: The IT-Emperor case , 2005, Internet High. Educ..

[27]  Steven A. Hillyard,et al.  Neural Basis of Superior Performance of Action Videogame Players in an Attention-Demanding Task , 2011, The Journal of Neuroscience.

[28]  V. Rideout,et al.  Generation M2: Media in the Lives of 8- to 18-Year-Olds , 2010 .

[29]  Brian G. Burton,et al.  Learning in 3D Virtual Environments: Collaboration and Knowledge Spirals , 2010 .

[30]  Erol Özçelik,et al.  The effect of competition on learning in games , 2015, Comput. Educ..

[31]  Jaejin Lee,et al.  What We Can Learn from the Data: A Multiple-Case Study Examining Behavior Patterns by Students with Different Characteristics in Using a Serious Game , 2016, Technol. Knowl. Learn..