Detecting the Moment of Learning

Intelligent tutors have become increasingly accurate at detecting whether a student knows a skill at a given time. However, these models do not tell us exactly at which point the skill was learned. In this paper, we present a machine-learned model that can assess the probability that a student learned a skill at a specific problem step (instead of at the next or previous problem step). Implications for knowledge tracing and potential uses in “discovery with models” educational data mining analyses are discussed, including analysis of which skills are learned gradually, and which are learned in “eureka” moments.

[1]  K. Duncker,et al.  On problem-solving , 1945 .

[2]  G. A. Ferguson,et al.  Statistical analysis in psychology and education , 1960 .

[3]  P. H. Jackson,et al.  Statistical analysis in psychology and education , 1960 .

[4]  Ira P. Goldstein,et al.  The genetic graph: a representation for the evolution of procedural knowledge , 1979 .

[5]  Tom M. Mitchell,et al.  Learning from Solution Paths: An Approach to the Credit Assignment Problem , 1982, AI Mag..

[6]  Allen and Rosenbloom Paul S. Newell,et al.  Mechanisms of Skill Acquisition and the Law of Practice , 1993 .

[7]  Ralph L. Rosnow,et al.  Essentials of Behavioral Research: Methods and Data Analysis , 1984 .

[8]  Stellan Ohlsson,et al.  Automated Cognitive Modeling , 1984, AAAI.

[9]  J. Metcalfe,et al.  Intuition in insight and noninsight problem solving , 1987, Memory & cognition.

[10]  K. VanLehn Mind Bugs: The Origins of Procedural Misconceptions , 1990 .

[11]  Joel D. Martin,et al.  Student assessment using Bayesian nets , 1995, Int. J. Hum. Comput. Stud..

[12]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[13]  Daniel L. Schwartz,et al.  Chapter 3: Rethinking Transfer: A Simple Proposal With Multiple Implications , 1999 .

[14]  Scott D. Brown,et al.  The power law repealed: The case for an exponential law of practice , 2000, Psychonomic bulletin & review.

[15]  Ian Witten,et al.  Data Mining , 2000 .

[16]  Mark A. Hall,et al.  Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning , 1999, ICML.

[17]  Kenneth R. Koedinger Toward Evidence for Instructional Design Principles: Examples from Cognitive Tutor Math 6 , 2002 .

[18]  K. VanLehn,et al.  Why Do Only Some Events Cause Learning During Human Tutoring? , 2003 .

[19]  Edward M. Bowden,et al.  New approaches to demystifying insight , 2005, Trends in Cognitive Sciences.

[20]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[21]  Valerie J. Shute,et al.  SMART: Student modeling approach for responsive tutoring , 1995, User Modeling and User-Adapted Interaction.

[22]  Vincent Aleven,et al.  Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor , 2006, Int. J. Artif. Intell. Educ..

[23]  Beverly Park Woolf,et al.  A Dynamic Mixture Model to Detect Student Motivation and Proficiency , 2006, AAAI.

[24]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[25]  Neil T. Heffernan,et al.  Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[26]  Neil T. Heffernan,et al.  Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required , 2006, Intelligent Tutoring Systems.

[27]  Ryan Shaun Joazeiro de Baker,et al.  Developing a generalizable detector of when students game the system , 2008, User Modeling and User-Adapted Interaction.

[28]  Kenneth R. Koedinger,et al.  Is Over Practice Necessary? - Improving Learning Efficiency with the Cognitive Tutor through Educational Data Mining , 2007, AIED.

[29]  Ryan Shaun Joazeiro de Baker,et al.  Modeling and understanding students' off-task behavior in intelligent tutoring systems , 2007, CHI.

[30]  Jack Mostow,et al.  How Who Should Practice: Using Learning Decomposition to Evaluate the Efficacy of Different Types of Practice for Different Types of Students , 2008, Intelligent Tutoring Systems.

[31]  Vincent Aleven,et al.  Improving Contextual Models of Guessing and Slipping with a Trucated Training Set , 2008, EDM.

[32]  Edward M. Bowden,et al.  The origins of insight in resting-state brain activity , 2008, Neuropsychologia.

[33]  Agneta Gulz,et al.  Catching Eureka on the Fly , 2008, AAAI Spring Symposium: Emotion, Personality, and Social Behavior.

[34]  John R Anderson,et al.  Using a model to compute the optimal schedule of practice. , 2008, Journal of experimental psychology. Applied.

[35]  Albert T. Corbett,et al.  Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments , 2008 .

[36]  Albert T. Corbett,et al.  Does Help Help? Introducing the Bayesian Evaluation and Assessment Methodology , 2008, Intelligent Tutoring Systems.

[37]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[38]  Carolina Ruiz,et al.  The Composition Effect: Conjuntive or Compensatory? An Analysis of Multi-Skill Math Questions in ITS , 2008, EDM.

[39]  Kenneth R. Koedinger,et al.  A Response Time Model For Bottom-Out Hints as Worked Examples , 2008, EDM.

[40]  Neil T. Heffernan,et al.  Does Self-Discipline impact students' knowledge and learning? , 2009, EDM.

[41]  Luo Si,et al.  Learning to Identify Students' Off-Task Behavior in Intelligent Tutoring Systems , 2009, AIED.

[42]  Arnon Hershkovitz,et al.  The Impact of Off-task and Gaming Behaviors on Learning: Immediate or Aggregate? , 2009, AIED.

[43]  Zachary A. Pardos,et al.  Determining the Significance of Item Order In Randomized Problem Sets , 2009, EDM.

[44]  Neil T. Heffernan,et al.  A Comparison of Traditional Homework to Computer-Supported Homework , 2009 .

[45]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[46]  Zachary A. Pardos,et al.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing , 2010, UMAP.

[47]  Ryan Shaun Joazeiro de Baker,et al.  Contextual Slip and Prediction of Student Performance after Use of an Intelligent Tutor , 2010, UMAP.

[48]  Neil T. Heffernan,et al.  Comparing Knowledge Tracing and Performance Factor Analysis by Using Multiple Model Fitting Procedures , 2010, Intelligent Tutoring Systems.

[49]  Kurt VanLehn,et al.  Do Micro-Level Tutorial Decisions Matter: Applying Reinforcement Learning to Induce Pedagogical Tutorial Tactics , 2010, Intelligent Tutoring Systems.

[50]  Zachary A. Pardos,et al.  Learning What Works in its from Non-Traditional Randomized Controlled Trial Data , 2010, Int. J. Artif. Intell. Educ..

[51]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[52]  Neil T. Heffernan,et al.  A Quasi-Experimental Evaluation of An On-Line Formative Assessment and Tutoring System , 2010 .

[53]  Thomas Bay,et al.  Gaming the system , 2011 .