The Sequence of Action Model: Leveraging the Sequence of Attempts and Hints

Intelligent Tutoring Systems (ITS) have been proven to be efficient providing student assistance and assessing their performance when they do their homework. Researchers have analyzed how students’ knowledge grows and predict their performance from within intelligent tutoring systems. Most of them focus on using correctness of the previous question or the number of hints and attempts students need to predict their future performance, but ignore the sequence of hints and attempts. In this research work, we build a Sequence of Actions (SOA) model taking advantage of the sequence of hints and attempts a student needed for the previous question to predict students’ performance. A two step modeling methodology is put forward in the work and is a combination of Tabling method and the Logistic Regression. We compared SOA with Knowledge Tracing (KT) and Assistance Model (AM) and combinations of SOA/AM and KT. The experimental results showed that the Sequence of Action model has reliably better predictive accuracy than KT and AM and its performance of prediction is improved after combining with KT.

[1]  Zachary A. Pardos,et al.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing , 2010, UMAP.

[2]  Zachary A. Pardos,et al.  Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm , 2010, EDM.

[3]  Zachary A. Pardos,et al.  Ensembling predictions of student knowledge within intelligent tutoring systems , 2011, UMAP'11.

[4]  Zachary A. Pardos,et al.  Does Time Matter? Modeling the Effect of Time with Bayesian Knowledge Tracing , 2011, EDM.

[5]  Kenneth R. Koedinger,et al.  Automated Student Model Improvement , 2012, EDM.

[6]  Ivon Arroyo,et al.  Tracing Knowledge and Engagement in Parallel in an Intelligent Tutoring System , 2014, EDM.

[7]  Brett van de Sande.  Applying three models of learning to individual student log data , 2013, EDM.

[8]  Joseph E. Beck,et al.  Engagement tracing: using response times to model student disengagement , 2005, AIED.

[9]  Leena M. Razzaq,et al.  Developing Fine-Grained Transfer Models in the ASSISTment System , 2007 .

[10]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[11]  Neil T. Heffernan,et al.  Extending the Assistance Model: Analyzing the Use of Assistance over Time , 2013, EDM.

[12]  Shou-De Lin,et al.  Feature Engineering and Classifier Ensemble for KDD Cup 2010 , 2010, KDD 2010.

[13]  mith,et al.  Evaluating Probabilistic Forecasts Using Information Theory , 2002 .

[14]  Jeremy P. Smith,et al.  Higher education outcomes, graduate employment and university performance indicators , 2004 .

[15]  Zachary A. Pardos,et al.  A Comparison of Error Metrics for Learning Model Parameters in Bayesian Knowledge Tracing , 2014, EDM.

[16]  Neil T. Heffernan,et al.  The impact of gaming ( ? ) on learning at the fine-grained level , 2010 .

[17]  Michel C. Desmarais,et al.  On the Faithfulness of Simulated Student Performance Data , 2010, EDM.

[18]  W. Sedlacek,et al.  Student Retention and The Use of Campus Facilities by Race , 2009 .

[19]  Albert T. Corbett,et al.  Student Modeling in the ACT Programming Tutor: Adjusting a Procedural Learning Model With Declarative Knowledge , 1997 .

[20]  Kenneth R. Koedinger,et al.  Unsupervised Discovery of Student Strategies , 2010, EDM.

[21]  Zlatko J. Kovacic,et al.  Early Prediction of Student Success: Mining Students Enrolment Data , 2010 .

[22]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[23]  N. Heffernan,et al.  Using HMMs and bagged decision trees to leverage rich features of user and skill from an intelligent tutoring system dataset , 2010 .

[24]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[25]  John Seely Brown,et al.  Intelligent Tutoring Systems , 2016, Lecture Notes in Computer Science.

[26]  Michael Jahrer,et al.  Collaborative Filtering Applied to Educational Data Mining , 2010 .

[27]  Albert T. Corbett,et al.  Modeling Student Knowledge: Cognitive Tutors in High School and College , 2000, User Modeling and User-Adapted Interaction.

[28]  Brett van de Sande,et al.  Properties of the Bayesian Knowledge Tracing Model , 2013, EDM 2013.

[29]  Zachary A. Pardos,et al.  Less is More: Improving the Speed and Prediction Power of Knowledge Tracing by Using Less Data , 2011, EDM.

[30]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[31]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[32]  Kenneth R. Koedinger,et al.  Estimating the benefits of student model improvements on a substantive scale , 2013, EDM.

[33]  Jennifer L. Molnar,et al.  The Sum Is Greater than the Parts , 2004 .

[34]  Vincent Aleven,et al.  Intelligent Tutoring Goes To School in the Big City , 1997 .

[35]  Yue Gong,et al.  Using Dirichlet priors to improve model parameter plausibility , 2009, EDM.

[36]  Neil T. Heffernan,et al.  Comparing Knowledge Tracing and Performance Factor Analysis by Using Multiple Model Fitting Procedures , 2010, Intelligent Tutoring Systems.

[37]  Yue Gong,et al.  Wheel-Spinning: Students Who Fail to Master a Skill , 2013, AIED.

[38]  Zachary A. Pardos,et al.  Towards Moment of Learning Accuracy , 2013, AIED Workshops.

[39]  Kenneth R. Koedinger,et al.  The Simple Location Heuristic is Better at Predicting Students' Changes in Error Rate Over Time Compared to the Simple Temporal Heuristic , 2011, EDM.

[40]  Markus H. Gross,et al.  Beyond Knowledge Tracing: Modeling Skill Topologies with Bayesian Networks , 2014, Intelligent Tutoring Systems.

[41]  Uri Wilensky,et al.  GasLab—an Extensible Modeling Toolkit for Connecting Micro-and Macro-properties of Gases , 1999 .

[42]  Neil T. Heffernan,et al.  Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required , 2006, Intelligent Tutoring Systems.

[43]  Albert T. Corbett,et al.  Does Help Help? Introducing the Bayesian Evaluation and Assessment Methodology , 2008, Intelligent Tutoring Systems.

[44]  Joseph E. Beck,et al.  Identifiability: A Fundamental Problem of Student Modeling , 2007, User Modeling.

[45]  Zachary A. Pardos,et al.  A Spectral Learning Approach to Knowledge Tracing , 2013, EDM.

[46]  R. Charles Murray,et al.  Reducing the Knowledge Tracing Space , 2009, EDM.

[47]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[48]  Ryan Shaun Joazeiro de Baker,et al.  Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction , 2005, Graphics Interface.

[49]  Ryan Shaun Joazeiro de Baker,et al.  Contextual Slip and Prediction of Student Performance after Use of an Intelligent Tutor , 2010, UMAP.

[50]  Neil T. Heffernan,et al.  The "Assistance" Model: Leveraging How Many Hints and Attempts a Student Needs , 2011, FLAIRS Conference.

[51]  Beverly Park Woolf,et al.  A Dynamic Mixture Model to Detect Student Motivation and Proficiency , 2006, AAAI.

[52]  R. Real,et al.  AUC: a misleading measure of the performance of predictive distribution models , 2008 .

[53]  Kurt VanLehn,et al.  The Andes Physics Tutoring System: Lessons Learned , 2005, Int. J. Artif. Intell. Educ..

[54]  Albert T. Corbett,et al.  Cognitive Computer Tutors: Solving the Two-Sigma Problem , 2001, User Modeling.

[55]  K. Koedinger,et al.  Fostering the Intelligent Novice: Learning From Errors With Metacognitive Tutoring , 2005 .

[56]  Zachary A. Pardos,et al.  Using Fine-Grained Skill Models to Fit Student Performance with , 2006 .

[57]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.