Student Skill Models in Adaptive Testing

This paper provides a common framework, a generic model, for Computerized Adaptive Testing (CAT) for different model types. We present question selection methods for CAT for this generic model. We use three different types of models, Item Response Theory, Bayesian Networks, and Neural Networks, that instantiate the generic model. We illustrate the usefulness of a special model condition ‐ the monotonicity ‐ and discuss its inclusion in these model types. With Bayesian networks we use specific type of learning using generalized linear models to ensure the monotonicity. We conducted simulated CAT tests on empirical data. Behavior of individual models was assessed based on these tests. The best performing model was the BN model constructed by a domain expert; its parameters were learned from data under the monotonicity condition.

[1]  Igor Aleksander,et al.  Introduction to Neural Computing , 1990 .

[2]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[3]  Participants' Reactions to Computerized Testing , 1988 .

[4]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[5]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[6]  Teuvo Kohonen,et al.  An introduction to neural computing , 1988, Neural Networks.

[7]  Jirí Vomlel,et al.  Bayesian Network Models for Adaptive Testing , 2015, BMA@UAI.

[8]  Linda C. van der Gaag,et al.  Learning from incomplete data in Bayesian networks with qualitative influences , 2016, Int. J. Approx. Reason..

[9]  Michael J. Culbertson,et al.  Bayesian Networks in Educational Assessment , 2016, Applied psychological measurement.

[10]  Marek J Druzdzel,et al.  Canonical Probabilistic Models for Knowledge Engineering , 2007 .

[11]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[12]  Jirí Vomlel,et al.  Bayesian Networks In Educational Testing , 2002, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[13]  José-Luis Pérez-de-la-Cruz,et al.  Bayesian networks for student model engineering , 2010, Comput. Educ..

[14]  Russell G. Almond,et al.  Graphical Models and Computerized Adaptive Testing , 1998 .

[15]  Anders L. Madsen,et al.  Bayesian networks and influence diagrams , 2007 .

[16]  Thomas G. Dietterich,et al.  Exploiting monotonicity via logistic regression in Bayesian network learning , 2013 .

[17]  S. Lauritzen.  The EM algorithm for graphical association models with missing data , 1995 .

[18]  Linda C. van der Gaag,et al.  Monotonicity in Bayesian Networks , 2004, UAI.

[19]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[20]  Finn Verner Jensen,et al.  Bayesian Networks and Influence Diagrams , 1997 .

[21]  Qiang Ji,et al.  Constrained Maximum Likelihood Learning of Bayesian Networks for Facial Action Recognition , 2008, ECCV.

[22]  Scott Tonidandel,et al.  Computer-adaptive testing: the impact of test characteristics on perceived performance and test takers' reactions. , 2002, The Journal of applied psychology.

[23]  Frank Rijmen,et al.  Bayesian networks with a logistic regression model for the conditional probabilities , 2008, Int. J. Approx. Reason..

[24]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[25]  S. Lipsitz,et al.  Missing-Data Methods for Generalized Linear Models , 2005 .

[26]  Russell G. Almond,et al.  Bayesian Networks in Educational Assessment , 2015 .

[27]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[28]  David J. Weiss,et al.  A Comparison of the Fairness of Adaptive and Conventional Testing Strategies. , 1978 .

[29]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[30]  อนุสรณ์ เกิดศรี,et al.  Elements of Adaptive Testing , 2015 .