THE EFFECT OF ITEM TYPE ON THE CONSEQUENCES OF CHANGING ANSWERS ON MULTIPLE CHOICE TESTS

Many research studies have consistently demonstrated the beneficial consequences of changing answers on objective tests (Bath, 1967; Foote & Belinky, 1972; Hill, 1937; Jarrett, 1948; Lamson, 1935; Lowe & Crawford, 1929; Mathews, 1929; Mueller & Shwedel, 1975; and Reile & Briggs, 1952), but only one study has examined the effects of the cognitive level of the test items as a factor in answer changing (Reiling & Taylor, 1972). Reiling & Taylor dichotomized their items as those that required analytical reasoning and those that did not. These classifications were not made by independent raters, but by the authors themselves; thus objectivity and reliability may be or would seem to be serious limitations in their study. The present study attempted to determine if there were differential rates of changing answers on test items according to the cognitive level of the items, and if the proportion of gains to losses across the items was different. Lower-order items were defined as those that tested knowledge or comprehension as defined by Bloom (1956). Higher-order items were those that tested cognitive processes such as application, synthesis, analysis, or evaluation. It was hypothesized that more gains would result from changes on higher-order items, since deliberating upon these items might enable the student to see where he or she had made a hasty or simplistic error; on lower-order items, the student generally either knows the answer or does not. Another aspect of changing which was investigated in this study was the degree to which net gain was related to overall knowledge. Several studies have shown that high-scoring students gain more points as a result of changing than do low-scoring students (Mallinson & Miller, 1956; Mathews, 1929; Mueller & Schwedel, 1975; Reile & Briggs, 1952; Reiling & Taylor, 1972). However, Mueller and Wasser (1977) have pointed out that in all of these studies the computed gain score was confounded by calculating the gain score after answer changing occurred. Thus, since answer changing tends to raise the total score, the reported relation between net gain and total score is only partly a result of the better students gaining more points than the poorer students. To eliminate this problem in the present study, two scores were computed for each individual: a prescore which reflected the score the student would have made if there had been no changes, and a post-score which represented the student's score as a result of changing.