Using Fine-Grained Skill Models to Fit Student Performance with

The ASSISTment online tutoring system was used by over 600 students during the school year 2004-2005. Each student used the system as part of their math classes 1-2 times a month, doing on average over 100+ state-test items, and getting tutored on the ones they got incorrect. The ASSISTment system has 4 different skill models, each at different grain-size involving 1, 5, 39 or 106 skills. Our goal in the paper is to develop a model that will predict whether a student will get correct a given item. We compared the performance of these models on their ability to predict a student state test score, after the state test was "tagged" with skills for the 4 models. The best fitting model was the 39 skill model, suggesting that using finer-grained skills models is useful to a point. This result is pretty much the same as that which was achieved by Feng, Heffernan, Mani, & Heffernan (in press), who were working simultaneously, but using mized-effect models instead of Bayes networks. We discuss reasons why the finest-grained model might not have been able to predict the data the best. Implications for large scale testing are discussed.