Statistical Modelling of Multiple-Choice and True/False Tests: Ways of Considering, and of Reducing, the Uncertainties Attributable To Guessing.

ABSTRACT Test unreliability due to guessing in multiple‐choice and true/false tests is analysed from first principles, and two new measures are described, with the intention that they should be of a sort that is easily communicated without reference to the underlying statistics. One measure is concerned with the resolution of defined levels of knowledge and the other with the probability of examinees being incorrectly ranked. How the measures decrease with both test length and number of response options per question is quantified. It is concluded that the results of many tests currently conducted are likely to be unacceptably unreliable. Procedures for increasing test reliability are discussed in a logical sequence intended to aid their understanding.