Automatically Detecting a Student's Preparation for Future Learning: Help Use is Key

We present an automated detector that can predict a student’s later performance on a paper test of preparation for future learning, a post-test involving learning new material to solve problems involving skills that are related but different than the skills studied in the tutoring system. This automated detector operates on features of student learning and behavior within a Cognitive Tutor for College Genetics. We show that this detector predicts preparation for future learning better than Bayesian Knowledge Tracing, a widely-used measure of student learning in Cognitive Tutors. We also find that this detector only needs limited amounts of student data (the first 20% of a student’s data from a tutor lesson) in order to achieve a substantial proportion of its asymptotic predictive power.

[1]  Michel C. Desmarais,et al.  Learned student models with item to item knowledge structures , 2006, User Modeling and User-Adapted Interaction.

[2]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[3]  Zachary A. Pardos,et al.  Ensembling predictions of student knowledge within intelligent tutoring systems , 2011, UMAP'11.

[4]  Daniel L. Schwartz,et al.  Rethinking transfer: A simple proposal with multiple implica-tions , 1999 .

[5]  Gautam Biswas,et al.  The Role of Feedback in Preparation for Future Learning: A Case Study in Learning by Teaching Environments , 2006, Intelligent Tutoring Systems.

[6]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[7]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[8]  Vincent Aleven,et al.  Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor , 2006, Int. J. Artif. Intell. Educ..

[9]  Ryan Shaun Joazeiro de Baker,et al.  Towards Predicting Future Transfer of Learning , 2011, AIED.

[10]  Joel D. Martin,et al.  Student assessment using Bayesian nets , 1995, Int. J. Hum. Comput. Stud..

[11]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[12]  Ryan Shaun Joazeiro de Baker,et al.  Developing a generalizable detector of when students game the system , 2008, User Modeling and User-Adapted Interaction.

[13]  Albert T. Corbett,et al.  A Cognitive Tutor for Genetics Problem Solving: Learning Gains and Student Modeling , 2010 .

[14]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[15]  F. Ramsey,et al.  The Statistical Sleuth , 1996 .

[16]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[17]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[18]  John R. Anderson,et al.  The Transfer of Cognitive Skill , 1989 .

[19]  Ryan Shaun Joazeiro de Baker,et al.  Modeling and understanding students' off-task behavior in intelligent tutoring systems , 2007, CHI.

[20]  Neil T. Heffernan,et al.  Detecting Learning Moment-by-Moment , 2011, Int. J. Artif. Intell. Educ..

[21]  J. Schofield.  Computers and classroom culture , 1995 .

[22]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[23]  Ryan Shaun Joazeiro de Baker,et al.  Contextual Slip and Prediction of Student Performance after Use of an Intelligent Tutor , 2010, UMAP.

[24]  Albert T. Corbett,et al.  Student Modeling in the ACT Programming Tutor: Adjusting a Procedural Learning Model With Declarative Knowledge , 1997 .

[25]  Neil T. Heffernan,et al.  Comparing Knowledge Tracing and Performance Factor Analysis by Using Multiple Model Fitting Procedures , 2010, Intelligent Tutoring Systems.

[26]  Ryan Shaun Joazeiro de Baker,et al.  Labeling Student Behavior Faster and More Precisely with Text Replays , 2008, EDM.

[27]  John R. Anderson,et al.  Student Modeling and Mastery Learning in a Computer-Based Proramming Tutor , 1992, Intelligent Tutoring Systems.

[28]  J. Ledolter.  The Statistical Sleuth , 2003 .

[29]  John R Anderson,et al.  Using a model to compute the optimal schedule of practice. , 2008, Journal of experimental psychology. Applied.

[30]  Kenneth R. Koedinger,et al.  A Response Time Model For Bottom-Out Hints as Worked Examples , 2008, EDM.

[31]  Daniel L. Schwartz,et al.  Preparing students for future learning with Teachable Agents , 2010 .