Advances in Assessment of Students' Intuitive Understanding of Physics through Gameplay Data

In this paper, the authors present advances in analyzing gameplay data as evidence of learning outcomes using computational methods of statistical analysis. These analyses were performed on data gathered from the SURGE learning environment Martinez-Garza, Clark, & Nelson, 2010. SURGE is a digital game designed to help students articulate their intuitive concepts of motion physics and organize them toward a more normative scientific understanding. Various recurring issues of assessment, which pervade assessment of learning in games more generally, prompted the authors to consider whether gameplay actions of learners in the context of the game can be analyzed to produce evidence of learning. The authors describe their approach to the analysis of game play in terms of qualitative assessment that the authors believe may lay the groundwork for the application of similar computationally-intensive techniques in other educational game contexts.

[1]  Alan M. Lesgold,et al.  Modeling the process of collaborative learning , 2007 .

[2]  M. Mayo.  Video Games: A Route to Large-Scale STEM Education? , 2009, Science.

[3]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[4]  Brian C. Nelson,et al.  SURGE: integrating Vygotsky's spontaneous and instructed concepts in a digital game? , 2010, ICLS.

[5]  Kurt Squire,et al.  Design Principles of Next-Generation Digital Gaming for Education. , 2003 .

[6]  Ramakrishnan Srikant,et al.  Mining sequential patterns , 1995, Proceedings of the Eleventh International Conference on Data Engineering.

[7]  Orit Parnafes.  What Does “Fast” Mean? Understanding the Physical World Through Computational Representations , 2007 .

[8]  James Paul Gee,et al.  What video games have to teach us about learning and literacy , 2007, CIE.

[9]  Brian C. Nelson,et al.  Exploring Newtonian mechanics in a conceptually-integrated digital game: Comparison of learning and affective outcomes for students in Taiwan and the United States , 2011, Comput. Educ..

[10]  Douglas B. Clark.  Longitudinal Conceptual Change in Students' Understanding of Thermal Equilibrium: An Examination of the Process of Conceptual Restructuring , 2006 .

[11]  Cen Li,et al.  Applying the Hidden Markov Model Methodology for Unsupervised Learning of Temporal Data , 2002 .

[12]  Brian C. Nelson,et al.  SURGE: intended and unintended science learning in games , 2010, ICLS.

[13]  Aroutis Foster,et al.  Games, Claims, Genres, and Learning , 2009 .

[14]  Douglas B. Clark,et al.  Games, Learning, and Society: Prediction and Explanation as Design Mechanics in Conceptually Integrated Digital Games to Help Players Articulate the Tacit Understandings They Build through Game Play , 2012 .

[15]  Ton de Jong,et al.  Scientific Discovery Learning with Computer Simulations of Conceptual Domains , 1998 .

[16]  John C. Nesbit,et al.  Sequential Pattern Analysis of Learning Logs: Methodology and Applications , 2010 .

[17]  Deborah Willis,et al.  Learning and Assessment: exposing the inconsistencies of theory and practice , 1993 .

[18]  Barbara Y. White,et al.  Sources of Difficulty in Understanding Newtonian Dynamics , 1983, Cogn. Sci..

[19]  Janyce Wiebe,et al.  A machine learning approach to assessing knowledge sharing during collaborative learning activities , 2002, CSCL.

[20]  Clive Sutton,et al.  Beliefs about science and beliefs about language , 1996 .

[21]  Gautam Biswas,et al.  Using Hidden Markov Models to Characterize Student Behaviors in Learning-by-Teaching Environments , 2008, Intelligent Tutoring Systems.

[22]  H. O'Neil,et al.  Classification of learning outcomes: evidence from the computer games literature , 2005 .

[23]  D. Hestenes,et al.  Force concept inventory , 1992 .