Modeling Diagnostic Assessments with Bayesian Networks

This paper defines Bayesian network models and examines their applications to IRT-based cognitive diagnostic modeling. These models are especially suited to building inference engines designed to be synchronous with the finer grained student models that arise in skills diagnostic assessment. Aspects of the theory and use of Bayesian network models are reviewed, as they affect applications to diagnostic assessment. The paper discusses how Bayesian network models are set up with expert information, improved and calibrated from data, and deployed as evidence-based inference engines. Aimed at a general educational measurement audience, the paper illustrates the flexibility and capabilities of Bayesian networks through a series of concrete examples, and without extensive technical detail. Examples are provided of proficiency spaces with direct dependencies among proficiency nodes, and of customized evidence models for complex tasks. This paper is intended to motivate educational measurement practitioners to learn more about Bayesian networks from the research literature, to acquire readily available Bayesian network software, to perform studies with real and simulated data sets, and to look for opportunities in educational settings that may benefit from diagnostic assessment fueled by Bayesian network modeling.

[1]  Russell G. Almond,et al.  Bayesian networks: A teacher's view , 2009, Int. J. Approx. Reason..

[2]  Kevin P. Murphy,et al.  Bayesian structure learning using dynamic programming and MCMC , 2007, UAI.

[3]  Perry L. Miller,et al.  ATTENDING: Critiquing a Physician's Management Plan , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[5]  Russell G. Almond,et al.  Models for Conditional Probability Tables in Educational Assessment , 2001, AISTATS.

[6]  Russell G. Almond,et al.  BAYESIAN NETWORK MODELS FOR LOCAL DEPENDENCE AMONG OBSERVABLE OUTCOME VARIABLES , 2006 .

[7]  Russell G. Almond,et al.  Graphical Models and Computerized Adaptive Testing , 1999 .

[8]  Russell G. Almond,et al.  Bayesian Network Models for Local Dependence Among Observable Outcome Variables , 2006 .

[9]  J. Templin,et al.  Skills Diagnosis Using IRT-Based Latent Class Models , 2007 .

[10]  R. Almond,et al.  Focus Article: On the Structure of Educational Assessments , 2003 .

[11]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[12]  Russell G. Almond,et al.  "I Can Name that Bayesian Network in Two Matrixes!" , 2007, BMA.

[13]  Russell G. Almond,et al.  AN ASSESSMENT FOR LEARNING SYSTEM CALLED ACED: DESIGNING FOR LEARNING EFFECTIVENESS AND ACCESSIBILITY , 2007 .

[14]  Russell G. Almond,et al.  Assessing Information and Communications Technology Literacy for Higher Education , 2004 .

[15]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.