Some Theoretical Properties of Mutual Information for Student Assessments in Intelligent Tutoring Systems

This paper presents recently discovered properties of mutual information between concepts and dichotomous test items. The properties generalize some common intuitions for comparing test items, and provide principled foundations for designing item-selection heuristics for student assessments in computer-assisted educational systems. We compare performance profiles achieved by systems that adopt mutual information and the Mahalanobis distance in the assessment task. Experimental results reveal that, all else being equal, the mutual information based methods offer better performance profiles. In addition, experimental results suggest that, when computing mutual information online is considered computationally costly, heuristics that are designed based on our theoretical findings serve as a good delegate for exact mutual information.

[1]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[2]  J. Rost,et al.  Applications of Latent Trait and Latent Class Models in the Social Sciences , 1998 .

[3]  Michael P. Wellman.  Fundamental Concepts of Qualitative Probabilistic Networks , 1990, Artif. Intell..

[4]  Russell G. Almond,et al.  DESIGN AND ANALYSIS IN A COGNITIVE ASSESSMENT , 2003 .

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Kurt VanLehn,et al.  Applications of simulated students: an exploration , 1994 .

[7]  David G. Stork,et al.  Pattern Classification , 1973 .

[8]  Joel D. Martin,et al.  J. Evaluation on an assessment system based on Bayesian student modeling , 1997 .

[9]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[10]  Russell G. Almond,et al.  Bayes Nets in Educational Assessment: Where the Numbers Come From , 1999, UAI.

[11]  Anthony E. Kelly,et al.  Attribute-mastery patterns from rule space as the basis for student models in algebra , 1994, Int. J. Hum. Comput. Stud..

[12]  Antonija Mitrovic,et al.  Optimising ITS Behaviour with Bayesian Networks and Decision Theory , 2001 .

[13]  Chao-Lin Liu.  Using mutual information for adaptive student assessments , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[14]  Jim E. Greer,et al.  Adaptive Assessment Using Granularity Hierarchies and Bayesian Nets , 1996, Intelligent Tutoring Systems.

[15]  S. Lauritzen.  The EM algorithm for graphical association models with missing data , 1995 .