A COMPARISON OF THE RELIABILITY AND VALIDITY OF TWO METHODS FOR ASSESSING PARTIAL KNOWLEDGE ON A MULTIPLE-CHOICE TEST

under one of three procedures. Results from those students administered the test under conventional directions provided a baseline for comparing, in terms of reliability and validity, the results from students who took the test under the differential weighting of response alternatives or the confidence testing instructions. Reliability was estimated by the split-half technique. Validity was estimated by correlating midterm test scores with scores on a final examination. This investigation provides some support for the contention that validity can be improved using more sophisticated testing techniques. Suggestions for the conduct of more definitive studies were offered. Tests presented in multiple-choice format pose a continuing problem for testers. The problem arises in connection with incorrect or omitted responses and consists of obtaining more information about an examinee's knowledge than that he could not derive the correct answer. Studies by Dressel and Schmid (1953) and Coombs, Milholland and Womer (1956) suggest that partial knowledge, that is, knowledge inadequate to produce a correct answer, is a measurable characteristic. Two kinds of procedures are typically employed, those that differentially weight the response alternatives and those that require examinees to report their confidence in the correctness of the response alternatives. Differential Weighting of Response Alternatives Differential weighting procedures deviate from the conventional way of scoring multiple-choice tests. Instead of scoring 1 for correct answers and 0 for incorrect and omitted responses, differential scoring weights are assigned to each response alternative to an item. A test score then consists of the sum of the weights of the response alternatives, both correct and incorrect, that the examinee chose in answering the test.