A Data-Driven Approach for Inferring Student Proficiency from Game Activity Logs

Student assessments are important because they allow collecting evidence about learning. However, time spent on evaluating students may be otherwise used for instructional activities. Computer-based learning platforms provide the opportunity for unobtrusively gathering students' digital learning footprints. This data can be used to track learning progress and make inference about student competencies. We present a novel data analysis pipeline, Student Proficiency Inferrer from Game data (SPRING), that allows modeling game playing behavior in educational games. Unlike prior work, SPRING is a fully data-driven method that does not require costly domain knowledge engineering. Moreover, it produces a simple interpretable model that not only fits the data but also predicts learning outcomes. We validate our framework using data collected from students playing 11 educational mini-games. Our results suggest that SPRING can predict math assessments accurately on withheld test data (Correlation=0.55, Spearman rho=0.51).

[1]  Biing-Hwang Juang,et al.  HMM clustering for connected word recognition , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[2]  Padhraic Smyth,et al.  Clustering Sequences with Hidden Markov Models , 1996, NIPS.

[3]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[4]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[5]  Padhraic Smyth,et al.  Clustering Using Monte Carlo Cross-Validation , 1996, KDD.

[6]  Mário A. T. Figueiredo,et al.  Similarity-Based Clustering of Sequences Using Hidden Markov Models , 2003, MLDM.

[7]  Michael I. Jordan,et al.  An HDP-HMM for systems with state persistence , 2008, ICML '08.

[8]  Russell G. Almond,et al.  You Can't Fatten A Hog by Weighing It - Or Can You? Evaluating an Assessment for Learning System Called ACED , 2008, Int. J. Artif. Intell. Educ..

[9]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[10]  M. Mayo.  Video Games: A Route to Large-Scale STEM Education? , 2009, Science.

[11]  V. Shute,et al.  Melding the Power of Serious Games and Embedded Assessment to Monitor and Foster Learning: Flow and Grow , 2009 .

[12]  Jonathan P. Rowe,et al.  Modeling User Knowledge with Dynamic Bayesian Networks in Interactive Narrative Environments , 2010, AIIDE.

[13]  Yun-En Liu,et al.  Gameplay analysis through state projection , 2010, FDG.

[14]  Jonathan P. Rowe,et al.  Integrating Learning and Engagement in Narrative-Centered Learning Environments , 2010, Intelligent Tutoring Systems.

[15]  Robert J. Mislevy,et al.  Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining , 2012, EDM 2012.

[16]  Bi-Ru Dai,et al.  Efficient Map/Reduce-Based DBSCAN Algorithm with Optimized Data Partition , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[17]  Wei-keng Liao,et al.  A new scalable parallel DBSCAN algorithm using the disjoint-set data structure , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[18]  Vincent Aleven,et al.  RumbleBlocks: Teaching science concepts to young children through a Unity game , 2012, 2012 17th International Conference on Computer Games (CGAMES).

[19]  Yun-En Liu,et al.  Predicting Player Moves in an Educational Game: A Hybrid Approach , 2013, EDM.

[20]  Vincent Aleven,et al.  Investigating the Solution Space of an Open-Ended Educational Game Using Conceptual Feature Extraction , 2013, EDM.

[21]  V. Shute,et al.  Stealth Assessment: Measuring and Supporting Learning in Video Games , 2013 .

[22]  Vincent Aleven,et al.  In search of learning: facilitating data analysis in educational games , 2013, CHI.

[23]  Zoran Popovic,et al.  Learning Individual Behavior in an Educational Game: A Data-Driven Approach , 2014, EDM.

[24]  Melissa Lazarín.  Testing Overload in America's Schools. , 2014 .

[25]  Rebecca Brown,et al.  Measuring Implicit Science Learning with Networks of Player-Game Interactions , 2015, CHI PLAY.

[26]  Stuart R. Kahl,et al.  Re-Balancing Assessment: Placing Formative and Performance Assessment at the Heart of Learning and Accountability. , 2015 .