Using Mutual Information for Adaptive Item Comparison and Student Assessment

The author analyzes properties of mutual information between dichotomous concepts and test items. The properties generalize some common intuitions about item comparison, and provide principled foundations for designing item-selection heuristics for student assessment in computer-assisted educational systems. The proposed item-selection strategies along with some common and conceivable methods, including mutual information-based methods and Euclidean and Mahalanobis distance-based methods, for student classification are evaluated in a simulation-based environment. The simulator relies on Bayesian networks for capturing the uncertainty in students’ responses to test items. Simulated results indicate that the heuristics built upon the theoretical properties offer satisfactory performance profiles for item selection, and, not surprisingly, mutual information-based methods offer better performance for the task of student classification than distance-based methods.