On the Importance of Random Error in the Study of Probability Judgment. Part I: New Theoretical Developments

Erev, Wallsten, and Budescu (1994) demonstrated that over- and underconfidence can be observed simultaneously in judgment studies, as a function of the method used to analyze the data. They proposed a general model to account for this apparent paradox, which assumes that overt responses represent true judgments perturbed by random error. To illustrate that the model reproduces the pattern of results, they assumed perfectly calibrated true opinions and a particular form (log-odds plus normally distributed error) of the model to simulate data from the full-range paradigm. In this paper we generalize these results by showing that they can be obtained with other instantiations of the same general model (using the binomial error distribution), and that they apply to the half-range paradigm as well. These results illustrate the robustness and generality of the model. They emphasize the need for new methodological approaches to determine whether observed patterns of over- or underconfidence represent real eAects or are primarily statistical artifacts. #1997 by John Wiley & Sons, Ltd.

[1]  M. L. Samuels,et al.  Statistical Reversion Toward the Mean: More Universal Than Regression Toward the Mean , 1991 .

[2]  Rami Zwick,et al.  Comparing the calibration and coherence of numerical and verbal probability judgments , 1993 .

[3]  Mats Björkman,et al.  Brunswikian and Thurstonian Origins of Bias in Probability Assessment: On the Interpretation of Stochastic Components of Judgment , 1997 .

[4]  Paul Slovic,et al.  Comparison of Bayesian and Regression Approaches to the Study of Information Processing in Judgment. , 1971 .

[5]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[6]  W. C. Howell,et al.  Uncertainty measurement: A cognitive taxonomy , 1978 .

[7]  David V. Budescu,et al.  On the Importance of Random Error in the Study of Probability Judgment. Part II: Applying the Stochastic Judgment Model to Detect Systematic Trends , 1997 .

[8]  Ward Edwards,et al.  Judgment under uncertainty: Conservatism in human information processing , 1982 .

[9]  G. Keren.  Calibration and probability judgements: Conceptual and methodological issues , 1991 .

[10]  M. Björkman.  Internal Cue Theory: Calibration and Resolution of Confidence in General Knowledge , 1994 .

[11]  A. Tversky,et al.  Support theory: A nonextensional representation of subjective probability. , 1994 .

[12]  A. Tversky,et al.  Variants of uncertainty , 1982, Cognition.

[13]  William C. Howell.  Compounding uncertainty from internal sources. , 1972 .

[14]  P. Juslin.  The Overconfidence Phenomenon as a Consequence of Informal Experimenter-Guided Selection of Almanac Items , 1994 .

[15]  David V. Budescu,et al.  A Comparative Study of Measures of Partial Knowledge in Multiple-Choice Tests , 1997 .

[16]  L. Thurstone.  A law of comparative judgment. , 1994 .

[17]  Willem A. Wagenaar,et al.  Calibration of probability assessments by professional blackjack dealers, statistical experts, and lay people , 1985 .

[18]  B. Fischhoff,et al.  Hypothesis Evaluation from a Bayesian Perspective. , 1983 .

[19]  I. Erev,et al.  Simultaneous Over- and Underconfidence: The Role of Error in Judgment Processes. , 1994 .

[20]  W. Edwards,et al.  Conservatism in a simple probability inference task. , 1966, Journal of experimental psychology.

[21]  T. Ll.  Psychophysical analysis. By L. L. Thurstone, 1927. , 1987 .

[22]  David V. Budescu,et al.  Encoding subjective probabilities: A psychological and psychometric review , 1983 .

[23]  T. Wallsten,et al.  Individual Decision Behavior , 1972 .

[24]  Handanhal Ravinder,et al.  Random error in additive decompositions of multiattribute utility , 1991 .

[25]  P. Schoemaker,et al.  Utility measurement: Signal, noise, and bias , 1992 .

[26]  Phillip E. Pfeifer,et al.  Are We Overconfident in the Belief That Probability Forecasters Are Overconfident , 1994 .

[27]  Jack B. Soll.  Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure☆ , 1996 .

[28]  A. H. Murphy,et al.  Diagnostic verification of probability forecasts , 1992 .

[29]  David L. Ronis,et al.  Components of probability judgment accuracy: Individual consistency and effects of subject matter and assessment method. , 1987 .

[30]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.