Refining Learning Maps with Data Fitting Techniques: Searching for Better Fitting Learning Maps

Learning sciences needs quantitative methods for comparing alternative theories of what students are learning. This study investigated the accuracy of a learning map and its utility to predict student responses. Our data included a learning map detailing a hierarchical prerequisite skill graph and student responses to questions developed specifically to assess the concepts and skills represented in the map. Each question aligned to one skill in the map, and each skill had one or more prerequisite skills. Our research goal was to seek improvements to the knowledge representation in the map using an iterative process. We applied a greedy iterative search algorithm to simplify the learning map by merging nodes together. Each successive merge resulted in a model with one skill less than the previous model. We share the results of the revised model, its reliability and reproducibility, and discuss the face validity of the most significant merges.

[1]  J. Sarama,et al.  Learning Trajectories in Mathematics Education , 2004 .

[2]  Tiffany Barnes,et al.  The Q-matrix Method: Mining Student Response Data for Knowledge , 2005 .

[3]  Zachary A. Pardos,et al.  Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm , 2010, EDM.

[4]  Mark J. Gierl,et al.  The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka's Rule-Space Approach , 2004 .

[5]  Jonathan E Freyberger,et al.  Using Association Rules to Guide a Search for Best Fitting Transfer Models of Student Learning , 2004 .

[6]  R. Real,et al.  AUC: a misleading measure of the performance of predictive distribution models , 2008 .

[7]  Kenneth R. Koedinger,et al.  Is Over Practice Necessary? - Improving Learning Efficiency with the Cognitive Tutor through Educational Data Mining , 2007, AIED.

[8]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[9]  Kenneth R. Koedinger,et al.  Generalized learning factors analysis: improving cognitive models with machine learning , 2009 .

[10]  Michel C. Desmarais,et al.  Bayesian Student Models Based on Item to Item Knowledge Structures , 2006, EC-TEL.

[11]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[12]  Kenneth R. Koedinger,et al.  A Machine Learning Approach for Automatic Student Model Discovery , 2011, EDM.

[13]  Jennifer M. Bay-Williams.  Teaching Student-Centered Mathematics: Developmentally Appropriate Instruction for Grades 3-5 (Volume II) (2nd Edition) (New 2013 Curriculum & Instruction Titles) , 2013 .

[14]  Mark J. Gierl,et al.  Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees' Cognitive Skills in Algebra on the SAT. , 2008 .

[15]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[16]  Thomas H. Hammond,et al.  Learning in Hierarchies , 2007 .

[17]  K. Tatsuoka.  RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[18]  Kenneth R. Koedinger,et al.  Learning Factors Transfer Analysis: Using Learning Curve Analysis to Automatically Generate Domain Models , 2009, EDM.

[19]  James M. Leszczenski,et al.  What’s in a Word? Extending Learning Factors Analysis to Model Reading Transfer , 2007 .

[20]  Yanbo Xu,et al.  Logistic Regression in a Dynamic Bayes Net Models Multiple Subskills Better! , 2011, EDM.