Predicting Correctness of Problem Solving in ITS with a Temporal Collaborative Filtering Approach

Collaborative filtering (CF) is a technique that utilizes how users are associated with items in a target application and predicts the utility of items for a particular user. Temporal collaborative filtering (temporal CF) is a time-sensitive CF approach that considers the change in user-item interactions over time. Despite its capability to deal with dynamic educational applications with rapidly changing user-item interactions, there is no prior research of temporal CF on educational tasks. This paper proposes a temporal CF approach to automatically predict the correctness of students' problem solving in an intelligent math tutoring system. Unlike traditional user-item interactions, a student may work on the same problem multiple times, and there are usually multiple interactions for a student-problem pair. The proposed temporal CF approach effectively utilizes information coming from multiple interactions and is compared to i) a traditional CF approach, ii) a temporal CF approach that uses a sliding-time-window but ignores old data and multiple interactions and iii) a combined temporal CF approach that uses a sliding-time-window together with multiple interactions. An extensive set of experiment results show that using multiple-interactions significantly improves the prediction accuracy while using sliding-time-windows doesn't make a significant difference.

[1]  Vincent Aleven,et al.  Toward Legal Argument Instruction with Graph Grammars and Collaborative Filtering Techniques , 2006, Intelligent Tutoring Systems.

[2]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[3]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[4]  Christian D. Schunn,et al.  Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives , 2006 .

[5]  Joseph E. Beck,et al.  High-Level Student Modeling with Machine Learning , 2000, Intelligent Tutoring Systems.

[6]  Xue Li,et al.  Time weight collaborative filtering , 2005, CIKM '05.

[7]  Linas Baltrunas,et al.  Towards Time-Dependant Recommendation based on Implicit Feedback , 2009 .

[8]  Luo Si,et al.  Predicting Correctness of Problem Solving from Low-level Log Data in Intelligent Tutoring Systems , 2009, EDM.

[9]  Gilles Bisson,et al.  Inducing High-Level Behaviors from Problem-Solving Traces Using Machine-Learning Tools , 2007, IEEE Intelligent Systems.

[10]  Licia Capra,et al.  Temporal collaborative filtering with adaptive neighbourhoods , 2009, SIGIR.

[11]  Niels Pinkwart,et al.  Using Collaborative Filtering Algorithms as eLearning Tools , 2009 .

[12]  Mimi Recker,et al.  Collaborative Information Filtering: A Review and an Educational Application , 2004, Int. J. Artif. Intell. Educ..

[13]  Alexey Tsymbal,et al.  The problem of concept drift: definitions and related work , 2004 .