Response selection strategies and realism of confidence judgments

Abstract Two studies were conducted to examine how response selection strategy is related to confidence ratings and to performance on general knowledge questions. In both studies subjects were asked to answer 80 general knowledge questions and to rate their confidence in the correctness of the answer selected. A pilot study, in which subjects thought aloud while answering general knowledge questions, was carried out to identify different response selection strategies. In the first study, 40 subjects were asked to indicate which of four strategies (immediate recognition, inference, intuition, or guessing) they used for selecting an answer. In Study 2, think aloud reports from 20 subjects were coded into the same four strategies. The distribution of strategies differed between the studies, but there were very similar relations among strategy, confidence, and correctness of answer in the two studies. Response selection strategy was related to correctness of answer when confidence was partialed out. More specifically, immediate recognition was associated with higher proportion correct than with the other strategies. It was also found that ratings of how difficult the knowledge questions were to fellow students of the subjects were on a much more realistic level than the confidence ratings were. It is concluded that people could improve their confidence judgments by taking into account (a) how difficult a question is to other people, and (b) the response selection strategy used for answering the question.