Comparison of similarity measures to differentiate players' actions and decision-making profiles in serious games analytics

Abstract Three Gameplay Action-Decision (GAD) profiles: Explorer, Fulfiller, and Quitter, have been identified based on individual's decision-making actions and navigational behaviors in situ serious games. The ability to profile trainees using serious games can yield new analytics and insights towards training and learning performance improvement, including the identification of weaknesses or potential training needs in the players towards adaptive training, and the creation of new diagnostics for prescriptive training, retraining, and remediation. Similarity measures of players' in-game course of actions (COAs) have been shown to be a viable approach in differentiating novices from experts in serious games. In this study, we examined and compared several popular similarity measures to see if any measure, or combination of measures, would be viable in differentiating players based on their GAD profiles in serious games. Our findings revealed that similarity measures, while significant in their predicting abilities individually, could gain more strength from one another in combination. More research is needed to create or develop new metrics and methods for players’ action and behavioral profiling in Serious Games Analytics.

[1]  R. Clark,et al.  Dangers in the evaluation of instructional media , 1992, Academic medicine : journal of the Association of American Medical Colleges.

[2]  Ruck Thawonmas,et al.  Analysis of revisitations in online games , 2011, Entertain. Comput..

[3]  P. Jaccard.  THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1 , 1912 .

[4]  Venu Govindaraju,et al.  Exploring Similarity Measures for Biometric Databases , 2005, AVBPA.

[5]  D. Ballabio,et al.  Classification tools in chemistry. Part 1: linear models. PLS-DA , 2013 .

[6]  Magy Seif El-Nasr,et al.  Visualizing and understanding players' behavior in video games: discovering patterns and supporting aggregation and comparison , 2011, SIGGRAPH 2011.

[7]  Mohamed A. El-Sayed,et al.  Study of Similarity Measures with Linear Discriminant Analysis for Face Recognition , 2015 .

[8]  R. Clark.  Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses , 1985 .

[9]  Monica W. Tracey,et al.  Does media affect learning: where are we now? , 2005 .

[10]  Michael Mateas,et al.  A data mining approach to strategy prediction , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[11]  Dirk Ifenthaler,et al.  Serious Games Analytics: Theoretical Framework , 2015 .

[12]  Anders Drachen,et al.  Methods for Evaluating Gameplay Experience in a Serious Gaming Context , 2010, Int. J. Comput. Sci. Sport.

[13]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[14]  Sung-Hyuk Cha.  Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions , 2007 .

[15]  Georg Zoeller.  Game Development Telemetry in Production , 2013, Game Analytics, Maximizing the Value of Player Data.

[16]  Gesellschaft für Klassifikation. Jahrestagung,et al.  From Data and Information Analysis to Knowledge Engineering, Proceedings of the 29th Annual Conference of the Gesellschaft für Klassifikation e.V., University of Magdeburg, March 9-11, 2005 , 2006, GfKl.

[17]  Simon Colton,et al.  Unsupervised Modeling of Player Style With LDA , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[18]  H. Jaap van den Herik,et al.  Games as personality profiling tools , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[19]  Robert Maribe Branch,et al.  Instructional Design: The ADDIE Approach , 2009 .

[20]  Christian Sebastian Loh,et al.  Performance metrics for serious games: Will the (real) expert please step forward? , 2013, Proceedings of CGAMES'2013 USA.

[21]  Arnond Anantachai,et al.  ASSESSING WHAT PLAYERS LEARNED IN SERIOUS GAMES: IN SITU DATA COLLECTION, INFORMATION TRAILS, AND QUANTITATIVE ANALYSIS , 2007 .

[22]  V. Thada,et al.  Comparison of Jaccard, Dice, Cosine Similarity Coefficient To Find Best Fitness Value for Web Retrieved Documents Using Genetic Algorithm , 2013 .

[23]  Christian Sebastian Loh,et al.  Information Trails: In-Process Assessment of Game-Based Learning , 2012 .

[24]  Luca Chittaro,et al.  VU-Flow: A Visualization Tool for Analyzing Navigation in Virtual Environments , 2006, IEEE Transactions on Visualization and Computer Graphics.

[25]  Magy Seif El-Nasr,et al.  Game Analytics , 2013, Springer London.

[26]  Federica Marcolin,et al.  Similarity Measures for Face Recognition , 2015 .

[27]  Darryl Charles,et al.  Player-Centred Game Design : Player Modelling and Adaptive Digital Games , 2005 .

[28]  Judy Goldsmith,et al.  Using Visual Analytics to Inform Rheumatoid Arthritis Patient Choices , 2015 .

[29]  Mark P. J. van der Loo,et al.  The stringdist Package for Approximate String Matching , 2014, R J..

[30]  Christian Sebastian Loh,et al.  Using Players' Gameplay Action-Decision Profiles to Prescribe Training: Reducing Training Costs with Serious Games Analytics , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[31]  Shanmugasundaram Hariharan,et al.  A Comparison of Similarity Measures for Text Documents , 2008, J. Inf. Knowl. Manag..

[32]  Stefan Rüping,et al.  Procurement Fraud Discovery using Similarity Measure Learning , 2008, Trans. Case Based Reason..

[33]  Marc-Olivier Killijian,et al.  Next place prediction using mobility Markov chains , 2012, MPM '12.

[34]  Christian Sebastian Loh,et al.  Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics , 2013, Education and Information Technologies.

[35]  Benno Stein,et al.  Near Similarity Search and Plagiarism Analysis , 2005, GfKl.

[36]  S. Dreyfus.  The Five-Stage Model of Adult Skill Acquisition , 2004 .

[37]  Rico Januar Sitorus,et al.  USE OF NARCOTICS SUPPORTS RISK BEHAVIORS , 2016 .

[38]  Alessandro Canossa,et al.  Defining personas in games using metrics , 2008, Future Play.

[39]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[40]  Christian Sebastian Loh,et al.  Predicting expert-novice performance as serious games analytics with objective-oriented and navigational action sequences , 2015, Comput. Hum. Behav..

[41]  Konrad Rieck,et al.  Similarity measures for sequential data , 2011, WIREs Data Mining Knowl. Discov..

[42]  Christian Sebastian Loh,et al.  Maximum Similarity Index (MSI): A metric to differentiate the performance of novices vs. multiple-experts in serious games , 2014, Comput. Hum. Behav..

[43]  Magy Seif El-Nasr,et al.  Methods for Game User Research: Studying Player Behavior to Enhance Game Design , 2013, IEEE Computer Graphics and Applications.

[44]  R. Clark.  Media will never influence learning , 1994 .

[45]  Dirk Ifenthaler,et al.  Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives , 2012 .

[46]  Gregory K. W. K. Chung.  Guidelines for the Design and Implementation of Game Telemetry for Serious Games Analytics , 2015 .

[47]  Károly Héberger,et al.  Why is Tanimoto index an appropriate choice for fingerprint-based similarity calculations? , 2015, Journal of Cheminformatics.

[48]  Christian Sebastian Loh,et al.  Measuring Expert Performance for Serious Games Analytics: From Data to Insights , 2015 .

[49]  Satoshi Kobayashi,et al.  A Similarity Measure for DNA Sequence Analysis Based on Locality , 1993 .

[50]  K. A. Ericsson,et al.  The making of an expert. , 2007, Harvard business review.

[51]  Dirk Ifenthaler,et al.  Serious games analytics : methodologies for performance measurement, assessment, and improvement , 2015 .

[52]  Robert Riener,et al.  The Role of Serious Games in Robot Exoskeleton-Assisted Rehabilitation of Stroke Patients , 2015 .

[53]  John Zacharias,et al.  Exploratory spatial behaviour in real and virtual environments , 2006 .