A Bayesian Diagnostic Algorithm for Student Modeling and its Evaluation

In this paper, we present a new approach to diagnosis in student modeling based on the use of Bayesian Networks and Computer Adaptive Tests. A new integrated Bayesian student model is defined and then combined with an Adaptive Testing algorithm. The structural model defined has the advantage that it measures students' abilities at different levels of granularity, allows substantial simplifications when specifying the parameters (conditional probabilities) needed to construct the Bayesian Network that describes the student model, and supports the Adaptive Diagnosis algorithm. The validity of the approach has been tested intensively by using simulated students. The results obtained show that the Bayesian student model has excellent performance in terms of accuracy, and that the introduction of adaptive question selection methods improves its behavior both in terms of accuracy and efficiency.

[1]  Antonija Mitrovic,et al.  Using Evaluation to Shape ITS Design: Results and Experiences with SQL-Tutor , 2002, User Modeling and User-Adapted Interaction.

[2]  Kurt VanLehn,et al.  Applications of simulated students: an exploration , 1994 .

[3]  José-Luis Pérez-de-la-Cruz,et al.  Internet based Evaluation System , 2000 .

[4]  José Manuel Gutiérrez,et al.  Expert Systems and Probabiistic Network Models , 1996 .

[5]  Jim Reye.  A Belief Net Backbone for Student Modelling , 1996, Intelligent Tutoring Systems.

[6]  Joel D. Martin,et al.  Student assessment using Bayesian nets , 1995, Int. J. Hum. Comput. Stud..

[7]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[8]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[9]  Stellan Ohlsson,et al.  Constraint-Based Student Modeling , 1994 .

[10]  José-Luis Pérez-de-la-Cruz,et al.  Adaptive Bayesian Networks for Multilevel Student Modelling , 2000, Intelligent Tutoring Systems.

[11]  Anthony Jameson,et al.  Numerical uncertainty management in user and student modeling: An overview of systems and issues , 2005, User Modeling and User-Adapted Interaction.

[12]  Jim Reye.  Two-Phase Updating of Student Models Based on Dynamic Belief Networks , 1998, Intelligent Tutoring Systems.

[13]  Antonija Mitrovic,et al.  Using a Probabilistic Student Model to Control Problem Difficulty , 2000, Intelligent Tutoring Systems.

[14]  Kurt VanLehn,et al.  Conceptual and Meta Learning During Coached Problem Solving , 1996, Intelligent Tutoring Systems.

[15]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[16]  Joseph E. Beck,et al.  Adaptation of Problem Presentation and Feedback in an Intelligent Mathematics Tutor , 1996, Intelligent Tutoring Systems.

[17]  Cristina Conati,et al.  POLA: a student modeling framework for Probabilistic On-Line Assessment of problem solving performance , 2001 .

[18]  Antonija Mitrovic,et al.  Experiences in Implementing Constraint-Based Modeling in SQL-Tutor , 1998, Intelligent Tutoring Systems.

[19]  R. Hambleton.  Principles and selected applications of item response theory. , 1989 .

[20]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[21]  David J. Weiss,et al.  A Comparison of IRT-Based Adaptive Mastery Testing and a Sequential Mastery Testing Procedure , 1983 .

[22]  David J. Weiss,et al.  APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS , 1984 .

[23]  Kurt VanLehn,et al.  Student Modeling from Conversational Test Data: A Bayesian Approach Without Priors , 1998, Intelligent Tutoring Systems.

[24]  Eugene Charniak,et al.  Bayesian Networks without Tears , 1991, AI Mag..

[25]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[26]  Valerie J. Shute,et al.  Intelligent Tutoring Systems: Past, Present, and Future. , 1994 .

[27]  Robert J. Mislevy,et al.  The Role of Probability-Based Inference in an Intelligent Tutoring System. , 1995 .

[28]  Jim E. Greer,et al.  Adaptive Assessment Using Granularity Hierarchies and Bayesian Nets , 1996, Intelligent Tutoring Systems.

[29]  Vicente Ponsoda Gil,et al.  Tests adaptativos informatizados , 1996 .

[30]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[31]  Cristina Conati,et al.  On-Line Student Modeling for Coached Problem Solving Using Bayesian Networks , 1997 .