The Effect of Model Granularity on Student Performance Prediction Using Bayesian Networks

A standing question in the field of Intelligent Tutoring Systems and User Modeling in general is what is the appropriate level of model granularity (how many skills to model) and how is that granularity derived? In this paper we will explore models with varying levels of skill generality (1, 5, 39 and 106 skill models) and measure the accuracy of these models by predicting student performance within our tutoring system called ASSISTment as well as their performance on a state standardized test. We employ the use of Bayes nets to model user knowledge and to use for prediction of student responses. Our results show that the finer the granularity of the skill model, the better we can predict student performance for our online data. However, for the standardized test data we received, it was the 39 skill model that performed the best. We view this as support for fine-grained skill models despite the finest grain model not predicting the state test scores the best.

[1]  K. Tatsuoka.  Toward an Integration of Item-Response Theory and Cognitive Error Diagnosis. , 1987 .

[2]  Zachary A. Pardos,et al.  Analyzing Fine-Grained Skill Models Using Bayesian and Mixed Effects Methods , 2007, AIED.

[3]  Zachary A. Pardos,et al.  Using Fine-Grained Skill Models to Fit Student Performance with , 2006 .

[4]  Cristina Conati,et al.  Using Bayesian Networks to Manage Uncertainty in Student Modeling , 2002, User Modeling and User-Adapted Interaction.

[5]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[6]  Paul Brna,et al.  User Modeling 2005, 10th International Conference, UM 2005, Edinburgh, Scotland, UK, July 24-29, 2005, Proceedings , 2005, User Modeling.

[7]  Brian W. Junker,et al.  Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system , 2006 .

[8]  B. Junker,et al.  Do Skills Combine Additively to Predict Task Di culty in Eighth-grade Mathematics ? , 2006 .

[9]  Carolina Ruiz,et al.  The Composition Effect: Conjuntive or Compensatory? An Analysis of Multi-Skill Math Questions in ITS , 2008, EDM.

[10]  Judy Kay,et al.  Artificial intelligence in education : shaping the future of learning through intelligent technologies , 2003 .

[11]  Jim E. Greer,et al.  Interacting with Inspectable Bayesian Student Models , 2004, Int. J. Artif. Intell. Educ..

[12]  Antonija Mitrovic,et al.  Using a Probabilistic Student Model to Control Problem Difficulty , 2000, Intelligent Tutoring Systems.

[13]  Robert J. Mislevy,et al.  The role of probability-based inference in an intelligent tutoring system , 2005, User Modeling and User-Adapted Interaction.

[14]  Bert Bredeweg,et al.  Student Modelling: The Key to Individualized Knowledge-Based Instruction , 2010, NATO ASI Series.

[15]  Mitsuru Ikeda,et al.  Proceedings of the 8th international conference on Intelligent Tutoring Systems , 2006 .

[16]  Gordon I. McCalla,et al.  Granularity-Based Reasoning and Belief Revision in Student Models , 1994 .

[17]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[18]  John R. Anderson,et al.  Student modeling in the ACT Programming Tutor. , 1995 .

[19]  Jim Reye,et al.  Student Modelling Based on Belief Networks , 2004, Int. J. Artif. Intell. Educ..

[20]  Tiffany Barnes,et al.  The Q-matrix Method: Mining Student Response Data for Knowledge , 2005 .

[21]  Neil T. Heffernan,et al.  Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required , 2006, Intelligent Tutoring Systems.

[22]  Ricardo Conejo,et al.  Introducing Prerequisite Relations in a Multi-layered Bayesian Student Model , 2005, User Modeling.

[23]  Kenneth R. Koedinger,et al.  Recasting the feedback debate: benefits of tutoring error detection and correction skills , 2003 .

[24]  Beverly Park Woolf,et al.  Inferring learning and attitudes from a Bayesian Network of log file data , 2005, AIED.

[25]  Murali Mani,et al.  Using Mixed-Effects Modeling to Compare Different Grain-Sized Skill Models , 2006 .