Should observed overconfidence be dismissed as a statistical artifact? Critique of Erev, Wallsten, and Budescu (1994)

I. Erev, T. S. Wallsten, and D. V. Budescu (1994) showed that the same probability judgment data can reveal both apparent overconfidence and underconfidence, depending on how the data are analyzed. To explain this seeming paradox, I. Erev et al. proposed a general model of judgment in which overt responses are related to underlying "true judgments" that are perturbed by error. A central conclusion of their work is that observed over- and underconfidence can be split into two components: (a) "true" over- and underconfidence and (b) "artifactual" over- and underconfidence due to error in judgment. It is argued in the present article that decomposing over- and underconfidence into true and artifactual components is inappropriate. The mistake stems from giving primacy to ambiguously defined model constructions (true judgments) over observed data.

[1]  Derek J. Koehler,et al.  Subjective Probability of Disjunctive Hypotheses: Local-Weight Models for Decomposition of Evidential Support , 1999, Cognitive Psychology.

[2]  Peter Ayton,et al.  How Real is Overconfidence , 1997 .

[3]  A. Tversky,et al.  Unpacking, repacking, and anchoring: advances in support theory. , 1997 .

[4]  I. Erev,et al.  Simultaneous Over- and Underconfidence: The Role of Error in Judgment Processes. , 1994 .

[5]  Mats Björkman,et al.  Brunswikian and Thurstonian Origins of Bias in Probability Assessment: On the Interpretation of Stochastic Components of Judgment , 1997 .

[6]  A. Tversky,et al.  Support theory: A nonextensional representation of subjective probability. , 1994 .

[7]  N. Sanders,et al.  Journal of behavioral decision making: "The need for contextual and technical knowledge in judgmental forecasting", 5 (1992) 39-52 , 1992 .

[8]  David V. Budescu,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part II: Applying the Stochastic Judgment Model to Detect Systematic Trends , 1997 .

[9]  Lyle Brenner,et al.  Overconfidence in Probability and Frequency Judgments: A Critical Examination , 1996 .

[10]  Phillip E. Pfeifer,et al.  Are We Overconfident in the Belief That Probability Forecasters Are Overconfident , 1994 .

[11]  Ido Erev,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part I: New Theoretical Developments , 1997 .

[12]  Jack B. Soll.  Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure☆ , 1996 .

[13]  Gideon Keren,et al.  On The Calibration of Probability Judgments: Some Critical Comments and Alternative Perspectives , 1997 .

[14]  A. Diederich,et al.  Evaluating and Combining Subjective Probability Estimates , 1997 .