Estimating Student Proficiency Using an Item Response Theory Model

Item Response Theory (IRT) models were investigated as a tool for student modeling in an intelligent tutoring system (ITS). The models were tested using real data of high school students using the Wayang Outpost, a computer-based tutor for the mathematics portion of the Scholastic Aptitude Test (SAT). A cross-validation framework was developed and three metrics to measure prediction accuracy were compared. The trained models predicted with 72% accuracy whether a student would answer a multiple choice problem correctly.

[1]  Susan E. Embretson,et al.  A multidimensional latent trait model for measuring learning and change , 1991 .

[2]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[3]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[4]  Beverly Park Woolf,et al.  Inferring Unobservable Learning Variables from Students' Help Seeking Behavior , 2004, Intelligent Tutoring Systems.

[5]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[6]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[7]  Antonija Mitrovic,et al.  Optimising ITS Behaviour with Bayesian Networks and Decision Theory , 2001 .

[8]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .

[9]  David L. Brown,et al.  Parameter Estimation Techniques for Modal Analysis , 1979 .

[10]  Hasmik Mehranian,et al.  Evaluating the Feasibility of Learning Student Models from Data , 2005 .

[11]  John R. Anderson,et al.  Cognitive Modeling and Intelligent Tutoring , 1990, Artif. Intell..

[12]  Beverly Park Woolf,et al.  Web-Based Intelligent Multimedia Tutoring for High Stakes Achievement Tests , 2004, Intelligent Tutoring Systems.

[13]  Joseph E. Beck,et al.  Engagement tracing: using response times to model student disengagement , 2005, AIED.

[14]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[15]  Tiffany Barnes,et al.  The q-matrix method of fault-tolerant teaching in knowledge assessment and data mining , 2003 .

[16]  R. Maruyama,et al.  On Test Scoring , 1927 .

[17]  L. Collins,et al.  Latent Class Models for Stage-Sequential Dynamic Latent Variables , 1992 .

[18]  Frank B. Baker,et al.  Item Response Theory : Parameter Estimation Techniques, Second Edition , 2004 .

[19]  F. Baker,et al.  Item response theory : parameter estimation techniques , 1993 .

[20]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .