Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics

The behavioral differences between expert and novice performance is a well-studied area in training literature. Advances in technology have made it possible to trace players’ actions and behaviors within an online gaming environment as user-generated data for performance assessment. In this study, we introduce the use of string similarity to differentiate likely-experts from a group of unknown performers (mixture of novices and experts) based on how similar their in-game actions are to that of experts. Our findings indicate that string similarity is viable as an empirical assessment method to differentiate likely-experts from novices and potentially useful as the first performance metric for Serious Games Analytics (SEGA).

[1]  Jonah E. Rockoff,et al.  provided that full credit, including © notice, is given to the source. Can You Recognize an Effective Teacher When You Recruit One? , 2008 .

[2]  Dirk Ifenthaler,et al.  Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives , 2012 .

[3]  Valerie J. Shute,et al.  Modeling, Assessing, and Supporting Key Competencies Within Game Environments , 2010 .

[4]  David H. Jonassen,et al.  Handbook of Task Analysis Procedures , 1989 .

[5]  Brian A. Jacob,et al.  Can You Recognize an Effective Teacher when You Recruit One , 2011 .

[6]  Charles Elkan,et al.  An Efficient Domain-Independent Algorithm for Detecting Approximately Duplicate Database Records , 1997, DMKD.

[7]  Natalia Karelaia,et al.  Skill, Luck, Overconfidence, and Risk Taking , 2008 .

[8]  Paul R. Ford,et al.  Expertise and expert performance in sport , 2008 .

[9]  Dirk Ifenthaler,et al.  Computer-Based Diagnostics and Systematic Analysis of Knowledge , 2010 .

[10]  Christian Sebastian Loh,et al.  Information Trails: In-Process Assessment of Game-Based Learning , 2012 .

[11]  F. Gobet,et al.  The Cambridge handbook of expertise and expert performance , 2006 .

[12]  D. Simons,et al.  The effects of video game playing on attention, memory, and executive control. , 2008, Acta psychologica.

[13]  Christian Sebastian Loh.  Improving the Impact and Return of Investment of Game-Based Learning , 2013, Int. J. Virtual Pers. Learn. Environ..

[14]  Nazlena Mohamad Ali,et al.  Designing Interactive Applications to Support Novel Activities , 2013, Adv. Hum. Comput. Interact..

[15]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[16]  Arnond Anantachai,et al.  ASSESSING WHAT PLAYERS LEARNED IN SERIOUS GAMES: IN SITU DATA COLLECTION, INFORMATION TRAILS, AND QUANTITATIVE ANALYSIS , 2007 .

[17]  Christian Sebastian Loh,et al.  Researching and Developing Serious Games as Interactive Learning Instructions , 2009, Int. J. Gaming Comput. Mediat. Simulations.

[18]  Magy Seif El-Nasr,et al.  Game Analytics , 2013, Springer London.

[19]  S. Dreyfus,et al.  A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition , 1980 .

[20]  Raymond J. Mooney,et al.  Adaptive duplicate detection using learnable string similarity measures , 2003, KDD '03.

[21]  Paul J. Feltovich,et al.  The Cambridge handbook of expertise and expert performance , 2006 .

[22]  Richard E. Clark,et al.  Confounding in Educational Computing Research , 1985 .

[23]  J. Wilson White,et al.  Discriminant function analysis in marine ecology: some oversights and their solutions , 2007 .

[24]  S. Dreyfus.  The Five-Stage Model of Adult Skill Acquisition , 2004 .

[25]  Magy Seif El-Nasr,et al.  Visualizing and understanding players' behavior in video games: discovering patterns and supporting aggregation and comparison , 2011, SIGGRAPH 2011.

[26]  Dragan Gasevic,et al.  Open Learning Analytics: an integrated modularized platform , 2011 .

[27]  Magy Seif El-Nasr,et al.  Benefits of Game Analytics: Stakeholders, Contexts and Domains , 2013, Game Analytics, Maximizing the Value of Player Data.

[28]  M. Stella Atkins,et al.  Eye gaze patterns differentiate novice and experts in a virtual laparoscopic surgery training environment , 2004, ETRA.

[29]  Jdm Underwood,et al.  Novice and expert performance with a dynamic control task: scanpaths during a computer game , 2005 .

[30]  Sushil J. Louis,et al.  Playing to learn: case-injected genetic algorithms for learning to play computer games , 2005, IEEE Transactions on Evolutionary Computation.

[31]  Riccardo Berta,et al.  Assessment in and of Serious Games: An Overview , 2013, Adv. Hum. Comput. Interact..

[32]  Melissa E. DeRosier,et al.  Toward effective game-based social skills tutoring for children: an evaluation of a social adventure game , 2010, FDG.

[33]  V. Catano,et al.  Personality as a predictor of professional behavior in dental school: comparisons with dental practitioners. , 2005, Journal of dental education.

[34]  Isabelle Bichindaritz,et al.  Case-Based Reasoning. Research and Development, 18th International Conference on Case-Based Reasoning, ICCBR 2010, Alessandria, Italy, July 19-22, 2010. Proceedings , 2010, ICCBR.

[35]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[36]  J. N. Buxton,et al.  Software engineering techniques , 1970 .

[37]  R. Rao.  From unstructured data to actionable intelligence , 2003 .

[38]  Patricia Hughes,et al.  Can we Improve on how we Select Medical Students? , 2002 .

[39]  Monica W. Tracey,et al.  Does media affect learning: where are we now? , 2005 .

[40]  Andreas Höfer,et al.  Exploratory Comparison of Expert and Novice Pair Programmers , 2008, Comput. Informatics.

[41]  Geoffrey H. Fletcher.  Don't Let Drip Damage Your District: An Array of New Initiatives Is Helping Make Student Data Interoperable , 2013 .

[42]  Ian D. Watson,et al.  Similarity-Based Retrieval and Solution Re-use Policies in the Game of Texas Hold'em , 2010, ICCBR.

[43]  William E. Winkler,et al.  The State of Record Linkage and Current Research Problems , 1999 .

[44]  Richard N. Van Eck.  Digital Game-Based Learning: It's Not Just the Digital Natives Who Are Restless. , 2006 .

[45]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .