Graphical Models and Computerized Adaptive Testing

Computerized adaptive testing (CAT) based on item response theory (IRT) is viewed from the perspective of graphical modeling (GM). GM provides methods for making inferences about multifaceted skills and knowledge, and for extracting data from complex performances. However, simply incorporating variables for all sources of variation is rarely successful. Thus, researchers must closely analyze the substance and structure of the problem to create more effective models. Researchers regularly employ sophisticated strategies to handle many sources of variability outside the IRT model. Relevant variables can play many roles without appearing in the operational IRT model per se, e.g., in validity studies, assembling tests, and constructing and modeling tasks. Some of these techniques are described from a GM perspective, as well as how to extend them to more complex assessment situations. Issues are illustrated in the context of language testing.

[1]  Robert J. Mislevy,et al.  Bayes modal estimation in item response models , 1986 .

[2]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[3]  Jay Magidson,et al.  Advances in factor analysis and structural equation models , 1979 .

[4]  Russell G. Almond,et al.  On Test Selection Strategies for Belief Networks , 1995, AISTATS.

[5]  Ross D. Shachter.  Evaluating Influence Diagrams , 1986, Oper. Res..

[6]  Eric T. Bradlow,et al.  A Bayesian random effects model for testlets , 1999 .

[7]  R. Freedle,et al.  The prediction of TOEFL reading item difficulty: implications for construct validity , 1993 .

[8]  Robert J. Mislevy,et al.  Integrating Cognitive and Psychometric Models to Measure Document Literacy. , 1990 .

[9]  R. Hambleton.  Principles and selected applications of item response theory. , 1989 .

[10]  Carl P. M. Rijkes,et al.  Loglinear multidimensional IRT models for polytomously scored items , 1988 .

[11]  Daniel O. Segall,et al.  Multidimensional adaptive testing , 1996 .

[12]  David J. Spiegelhalter,et al.  Bayesian analysis in expert systems , 1993 .

[13]  Willem J. van der Linden,et al.  Multidimensional Adaptive Testing with a Minimum Error-Variance Criterion , 1999 .

[14]  Eric Horvitz,et al.  An Approximate Nonmyopic Computation for Value of Information , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Michael I. Jordan.  Graphical Models , 2003 .

[16]  Robert J. Mislevy,et al.  How to Equate Tests With Little or No Data , 1993 .

[17]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[18]  K. Tatsuoka.  Toward an Integration of Item-Response Theory and Cognitive Error Diagnosis. , 1987 .

[19]  R. Owen,et al.  A Bayesian Sequential Procedure for Quantal Response in the Context of Adaptive Mental Testing , 1975 .

[20]  J. Albert.  Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling , 1992 .

[21]  Donald E. Powers,et al.  The Relationship of Content Characteristics of GRE Analytical Reasoning Items to Their Difficulties and Discriminations , 1989 .

[22]  Martha L. Stocking,et al.  A Method for Severely Constrained Item Selection in Adaptive Testing , 1992 .

[23]  Martijn P. F. Berger,et al.  A Review of Selection Methods for Optimal Test Design. Research Report 94-4. , 1994 .

[24]  H. Widdowson,et al.  Teaching Language as Communication , 1979 .

[25]  D. Schum.  The Evidential Foundations of Probabilistic Reasoning , 1994 .

[26]  Lyle F. Bachman,et al.  语言测试实践 = Language testing in practice , 1998 .

[27]  Russell G. Almond.  Graphical belief modeling , 1995 .

[28]  Raymond J. Adams,et al.  The Multidimensional Random Coefficients Multinomial Logit Model , 1997 .

[29]  Lyle F. Bachman.  语言测试要略 = Fundamental considerations in language testing , 1990 .

[30]  Patrick Tapsfield,et al.  The British Army Recruit Battery Goes Operational: From Theory to Practice in Computer‐Based Testing Using Item‐Generation Techniques , 1995 .