Problem-Problem Solver Characteristics Affectingthe Calibration of Judgments

Abstract The purpose of this paper is to examine factors that affect the calibration of judgments by systematically comparing experts’ judgments to novices’ when solving a complex, real-world problem that varies in its initial characteristics. Calibration in this context refers to the proportion of times decision makers provide a range about their best estimates that includes the actual outcome. We found that experts specify a narrower range and provide more accurate best estimates than novices. But their tighter ranges are not justified by their greater accuracy: they are less likely to encompass the actual outcome than are novices. However, this effect is attenuated when solving more complex problems. Novices apparently underestimate the complexity of difficult problems, hence the accuracy of their best estimates decreases as does the width of their ranges, resulting in worse calibration. The performance of experts was not significantly different across problem solving conditions. Both groups provided asymmetrical ranges about their best estimates, which suggests they account for the effect of subproblem dependencies.