Clustering Students to Generate an Ensemble to Improve Standard Test Score Predictions

In typical assessment student are not given feedback, as it is harder to predict student knowledge if it is changing during testing. Intelligent Tutoring systems, that offer assistance while the student is participating, offer a clear benefit of assisting students, but how well can they assess students? What is the trade off in terms of assessment accuracy if we allow student to be assisted on an exam. In a prior study, we showed the assistance with assessments quality to be equal. In this work, we introduce a more sophisticated method by which we can ensemble together multiple models based upon clustering students. We show that in fact, the assessment quality as determined by the assistance data is a better estimator of student knowledge. The implications of this study suggest that by using computer tutors for assessment, we can save much instructional time that is currently used for just assessment.

[1]  G.,et al.  Ensemble Methods in Machine , 2007 .

[2]  Ann L. Brown,et al.  Dynamic Assessment: One Approach and Some Initial Data. Technical Report No. 361. , 1985 .

[3]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[4]  N. Heffernan,et al.  Using HMMs and bagged decision trees to leverage rich features of user and skill from an intelligent tutoring system dataset , 2010 .

[5]  OpitzDavid,et al.  Popular ensemble methods , 1999 .

[6]  Ana I. González Acuña An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, Boosting, and Randomization , 2012 .

[7]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[8]  Thomas G. Dietterich An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[9]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[10]  Patricia Charlton,et al.  Conference on Artificial Intelligence in Education , 2009 .

[11]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[12]  Neil T. Heffernan,et al.  Can We Get Better Assessment From A Tutoring System Compared to Traditional Paper Testing? Can We Have Our Cake (Better Assessment) and Eat It too (Student Learning During the Test)? , 2010, EDM.

[13]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[14]  Lynn S. Fuchs,et al.  Dynamic Assessment as Responsiveness to Intervention; a Scripted Protocol to Identify Young At-Risk Readers , 2007 .

[15]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[16]  L. Fuchs,et al.  Dynamic Assessment of Algebraic Learning in Predicting Third Graders' Development of Mathematical Problem Solving. , 2008, Journal of educational psychology.

[17]  T. Schack,et al.  Dynamic testing , 2003 .

[18]  Peter Tiño,et al.  Managing Diversity in Regression Ensembles , 2005, J. Mach. Learn. Res..