Modeling common misconceptions in learning process data

Student mistakes are often not random but, rather, reflect thoughtful yet incorrect strategies. In order for educational technologies to make full use of students' performance data to estimate the knowledge of a student, it is important to model not only the conceptions but also the misconceptions that a student's particular pattern of successes and errors may indicate. The student models that drive the "outer loop" of Intelligent Tutoring Systems typically do not represent or track misconceptions. Here, we present a method of representing misconceptions in the Knowledge Component models, or Q-Matrices, that are used by student models to estimate latent knowledge. We show, in a case study on a fraction arithmetic dataset, that incorporating a misconception into the Knowledge Component model dramatically improves the overall model's fit to data. We also derive qualitative insights from comparing predicted learning curves across models that incorporate varying misconception-related parameters. Finally, we show that the inclusion of a misconception in the Knowledge Component model can yield individual student estimates of misconception strength that are significantly correlated with out-of-tutor measures of student errors.

[1]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[2]  Kenneth R. Koedinger,et al.  Using Data-Driven Discovery of Better Student Models to Improve Student Learning , 2013, AIED.

[3]  Kenneth R. Koedinger,et al.  Automated Student Model Improvement , 2012, EDM.

[4]  Kurt VanLehn,et al.  The Behavior of Tutoring Systems , 2006, Int. J. Artif. Intell. Educ..

[5]  Tiffany Barnes,et al.  The Q-matrix Method: Mining Student Response Data for Knowledge , 2005 .

[6]  Kenneth R. Koedinger,et al.  Variations in Learning Rate: Student Classification Based on Systematic Residual Error Patterns across Practice Opportunities. , 2015, EDM 2015.

[7]  Richard E. Mayer,et al.  Using erroneous examples to improve mathematics learning with a web-based tutoring system , 2014, Comput. Hum. Behav..

[8]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[9]  Patricia A. Alexander,et al.  Evaluating Students’ Errors on Cognitive Tasks: Applications of Polytomous Item Response Theory and Log-Linear Modeling , 1994 .

[10]  P. Fayers Item Response Theory for Psychologists , 2004, Quality of Life Research.

[11]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[12]  K. Crowley,et al.  The microgenetic method. A direct means for studying cognitive development. , 1991, The American psychologist.

[13]  Rochel Gelman,et al.  Enabling constraints for cognitive development and learning: Domain specificity and epigenesis. , 1998 .

[14]  Yujing Ni,et al.  Teaching and Learning Fraction and Rational Numbers: The Origins and Implications of Whole Number Bias , 2005 .

[15]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[16]  H. Akaike Prediction and Entropy , 1985 .

[17]  K. Tatsuoka RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[18]  Kenneth R. Koedinger,et al.  Comparing Two IRT Models for Conjunctive Skills , 2008, Intelligent Tutoring Systems.

[19]  Albert T. Corbett,et al.  The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning , 2012, Cogn. Sci..

[20]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.