Using Mixed-Effects Modeling to Compare Different Grain-Sized Skill Models

Most assessments, like the math subtest of the SAT or the GRE, are unidimensional, in that they treat all questions on the test as sampling a single underlying “skill”. Can we predict state tests scores better if we tag the questions with fine-grained models the skills needed? Psychometricians don’t do this presumably because they don’t get better fitting model, for a variety of reasons. We are investigating if we can do better prediction with finer-grained skill models. Our result gave a confirmative answer to this

[1]  B. Junker,et al.  Do Skills Combine Additively to Predict Task Di culty in Eighth-grade Mathematics ? , 2006 .

[2]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[3]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[4]  K. Koedinger,et al.  Automating Cognitive Model Improvement by A * Search and Logistic Regression , 2005 .

[5]  John R. Anderson,et al.  Student modeling in the ACT Programming Tutor. , 1995 .

[6]  V. Carey,et al.  Mixed-Effects Models in S and S-Plus , 2001 .

[7]  J. Singer,et al.  Applied Longitudinal Data Analysis , 2003 .

[8]  Donald Hedeker,et al.  Longitudinal Data Analysis , 2006 .

[9]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .

[10]  C. Anderson,et al.  Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence , 2005 .

[11]  Zachary A. Pardos,et al.  Using Fine-Grained Skill Models to Fit Student Performance with , 2006 .

[12]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[13]  Brian W. Junker,et al.  Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system , 2006 .

[14]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[15]  Mitsuru Ikeda,et al.  Proceedings of the 8th international conference on Intelligent Tutoring Systems , 2006 .

[16]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[17]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[18]  E. Muraki,et al.  Full-Information Item Factor Analysis , 1988 .

[19]  Tiffany Barnes,et al.  The Q-matrix Method: Mining Student Response Data for Knowledge , 2005 .

[20]  Neil T. Heffernan,et al.  Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required , 2006, Intelligent Tutoring Systems.

[21]  K. Tatsuoka.  Toward an Integration of Item-Response Theory and Cognitive Error Diagnosis. , 1987 .

[22]  Neil T. Heffernan,et al.  Scaffolding vs. Hints in the Assistment System , 2006, Intelligent Tutoring Systems.