Calibration, additivity, and source independence of probability judgments in general knowledge and sensory discrimination tasks

Abstract Predictions for probability judgments in a general knowledge task were derived from the combined error model ( Juslin, Olsson, & Bjorkman, 1997 ) and predictions for probability judgments in a sensory discrimination task were derived from the sensory sampling model ( Juslin & Olsson, 1997 ). As predicted, an experiment suggested: (a) good calibration with full-range probability assessments for general knowledge tasks, but underconfidence for sensory discrimination, (b) more deviations from additivity in sensory discrimination, and (c) a preference to bet specifically on general knowledge or sensory tasks depending on the assessed probability, violating source-independence. The quantitative fit of the two models appropriately distinguished between the two task contents, but both models substantially under-predicted the overconfidence observed with interval estimation.

[1]  Phillip E. Pfeifer,et al.  Are We Overconfident in the Belief That Probability Forecasters Are Overconfident , 1994 .

[2]  P. Juslin,et al.  Visual perception of dynamic properties: cue heuristics versus direct-perceptual competence. , 2000, Psychological review.

[3]  P. Juslin,et al.  Format dependence in subjective probability calibration , 1999 .

[4]  L. J. Savage,et al.  The Foundations of Statistics , 1955 .

[5]  Claudia González-Vallejo,et al.  Statement Verification: A Stochastic Model of Judgment and Response. , 1994 .

[6]  R. Hogarth,et al.  Ambiguity and Uncertainty in Probabilistic Inference. , 1985 .

[7]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[8]  D. Ellsberg.  Decision, probability, and utility: Risk, ambiguity, and the Savage axioms , 1961 .

[9]  Gerd Gigerenzer,et al.  How to Improve Bayesian Reasoning Without Instruction: Frequency Formats , 1995 .

[10]  I. Erev,et al.  Simultaneous Over- and Underconfidence: The Role of Error in Judgment Processes. , 1994 .

[11]  B. Brehmer.  The psychology of linear judgement models , 1994 .

[12]  C. Mckenzie.  Taking into account the strength of an alternative hypothesis. , 1998 .

[13]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[14]  Juslin,et al.  The Calibration Issue: Theoretical Comments on Suantak, Bolger, and Ferrell (1996). , 1998, Organizational behavior and human decision processes.

[15]  D. Koehler,et al.  Probability judgment in three-category classification learning. , 2000, Journal of Experimental Psychology. Learning, Memory and Cognition.

[16]  P. Juslin,et al.  Thurstonian and Brunswikian origins of uncertainty in judgment: a sampling model of confidence in sensory discrimination. , 1997, Psychological review.

[17]  Richard A. Block,et al.  Overconfidence in estimation: Testing the anchoring-and-adjustment hypothesis , 1991 .

[18]  S. Stoness,et al.  Proceedings of the Twenty First Annual Conference of the Cognitive Science Society , 1999 .

[19]  F. Ramsey.  The Foundations of Mathematics and Other Logical Essays , 2001 .

[20]  M. Björkman.  Internal Cue Theory: Calibration and Resolution of Confidence in General Knowledge , 1994 .

[21]  A. Tversky,et al.  Unpacking, repacking, and anchoring: advances in support theory. , 1997 .

[22]  J. Baranski,et al.  The calibration and resolution of confidence in perceptual judgments , 1994, Perception & psychophysics.

[23]  Derek J. Koehler,et al.  The enhancement effect in probability judgment , 1997 .

[24]  Kenneth R. Hammond,et al.  Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice , 2000 .

[25]  Jack B. Soll,et al.  Overconfidence: It Depends on How, What, and Whom You Ask. , 1999, Organizational behavior and human decision processes.

[26]  W. Ferrell,et al.  A model for realism of confidence judgments: Implications for underconfidence in sensory discrimination , 1995, Perception & psychophysics.

[27]  D Vickers,et al.  Decision making and memory: a critique of Juslin and Olsson's (1997) sampling model of sensory discrimination. , 2001, Psychological review.

[28]  P. Juslin,et al.  Realism of confidence in sensory discrimination: The underconfidence phenomenon , 1993, Perception & psychophysics.

[29]  P. Juslin.  The Overconfidence Phenomenon as a Consequence of Informal Experimenter-Guided Selection of Almanac Items , 1994 .

[30]  Robyn M. Dawes,et al.  The False Consensus Effect and Overconfidence: Flaws in Judgment or Flaws in How We Study Judgment? , 1996 .

[31]  Jack B. Soll.  Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure☆ , 1996 .

[32]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[33]  Mats Björkman,et al.  Brunswikian and Thurstonian Origins of Bias in Probability Assessment: On the Interpretation of Stochastic Components of Judgment , 1997 .

[34]  P. Juslin,et al.  Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. , 2000, Psychological review.

[35]  A. Tversky,et al.  Support theory: A nonextensional representation of subjective probability. , 1994 .

[36]  When learning is detrimental: SESAM and outcome feedback , 1999 .

[37]  A. Tversky,et al.  Preference and belief: Ambiguity and competence in choice under uncertainty , 1991 .

[38]  P. Juslin,et al.  Judgment and decision making: Neo-Brunswikian and process tracing approaches , 1999 .

[39]  H. Olsson,et al.  Underconfidence in sensory discrimination: The interaction between experimental setting and response strategies , 1996, Perception & psychophysics.

[40]  William R. Ferrell,et al.  A model of calibration for subjective probabilities , 1980 .

[41]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[42]  J. Baranski,et al.  Context, feedback, and the calibration and resolution of confidence in perceptual judgments. , 1997, American Journal of Psychology.