A case for data-driven testing

This paper describes a novel approach to the on-line assessment of large groups of students, in which it may be desirable to maintain common questions between the groups. It is clear from the literature that computer based assessment has the potential to dramatically reduce the effort involved in testing and marking however problems arise where the cohort of students is larger than the number of available computers. However, the opposite situation is often true in practice, due to the perceived need to design multiple tests. The solution described here uses a small computer laboratory (20 machines) to administer a test to a series of groups of students in existing lab sessions. Each group receives the same set of questions but the data to which the questions apply, and hence the test answers, vary from group to group. The data from tests that have been applied to students is analysed to determine whether discussions with early candidates have influenced the performance of students in later testing sessions.