Taming Log Files From Game/Simulation‐Based Assessments: Data Models and Data Analysis Tools

Extracting information efficiently from game/simulation-based assessment (G/SBA) logs requires two things: a well-structured log file and a set of analysis methods. In this report, we propose a generic data model specified as an extensible markup language (XML) schema for the log files of G/SBAs. We also propose a set of analysis methods for identifying useful information from the log files and implement the methods in a package in the Python programming language, glassPy. We demonstrate the data model and glassPy with logs from a game-based assessment, SimCityEDU.

[1]  Chen Li,et al.  Classification of Writing Patterns Using Keystroke Logs , 2016 .

[2]  Robert J. Mislevy,et al.  Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining , 2012, EDM 2012.

[3]  Jiangang Hao,et al.  Analyzing Process Data from Game/Scenario-Based Tasks: An Edit Distance Approach , 2015, EDM 2015.

[4]  Yoav Bergner,et al.  Visualization and Confirmatory Clustering of Sequence Data from a Simulation-Based Assessment Task , 2014, EDM.

[5]  Gregory K. W. K. Chung,et al.  Identifying Key Features of Student Performance in Educational Video Games and Simulations through Cluster Analysis , 2012, EDM 2012.

[6]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[7]  Roy Williams.  VOTable: A Proposed XML Format for Astronomical Tables , 2002 .

[8]  V. Elizabeth Owen,et al.  Game-based assessment: an integrated model for capturing evidence of learning in play , 2014, Int. J. Learn. Technol..

[9]  Celestino Rodríguez Pérez.  Educational measurement: From Foundations to Future Wells, C. S., & Faulkner-Bond, M. (Eds.) (2016). New York: The Guilford Press , 2017 .

[10]  Robert J. Mislevy,et al.  Implications of Evidence‐Centered Design for Educational Testing , 2007 .

[11]  David Gibson,et al.  Theoretical Considerations for Game-Based e-Learning Analytics , 2015 .

[12]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[13]  Virginia Teller.  Review of Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition by Daniel Jurafsky and James H. Martin. Prentice Hall 2000. , 2000 .