Effects of Amount of Information on Judgment Accuracy and Confidence

When a person evaluates his or her confidence in a judgment, what is the effect of receiving more judgment-relevant information? We report three studies that show when judges receive more information, their confidence increases more than their accuracy, producing substantial confidence-accuracy discrepancies. Our results suggest that judges do not adjust for the cognitive limitations that reduce their ability to use additional information effectively. We place these findings in a more general framework of understanding the cues to confidence that judges use and how those cues relate to accuracy and calibration.

[1]  B. Brehmer,et al.  Human judgment : the SJT view , 1988 .

[2]  Brad M. Barber,et al.  Boys Will Be Boys: Gender, Overconfidence, and Common Stock Investment , 1998 .

[3]  J. Edward Russo,et al.  Winning Decisions: Getting It Right the First Time , 2001 .

[4]  Steven J. Sherman,et al.  Social explanation: The role of timing, set, and recall on subjective likelihood estimates. , 1983 .

[5]  P. Juslin,et al.  Format dependence in subjective probability calibration , 1999 .

[6]  Derek J. Koehler,et al.  Heuristics and Biases: The Calibration of Expert Judgment: Heuristics and Biases Beyond the Laboratory , 2002 .

[7]  L. Ross,et al.  The role of construal processes in overconfident predictions about the self and others. , 1990, Journal of personality and social psychology.

[8]  J. E. Russo,et al.  Biased interpretation of evidence by mock jurors. , 2001, Journal of experimental psychology. Applied.

[9]  P. Thagard,et al.  Coherence in Thought and Action , 2000 .

[10]  Kenneth R. Hammond,et al.  Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice , 2000 .

[11]  T. Trabasso,et al.  Constructing inferences during narrative text comprehension. , 1994 .

[12]  N. Pennington,et al.  Evidence evaluation in complex decision making. , 1986 .

[13]  N. Schwarz.  Metacognitive Experiences in Consumer Judgment and Decision Making , 2004 .

[14]  G. Pallier,et al.  Gender Differences in the Self-Assessment of Accuracy on Cognitive Tasks , 2003 .

[15]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[16]  N. Pennington,et al.  Explanation-based decision making: effects of memory structure on judgment , 1988 .

[17]  Jack B. Soll,et al.  Overconfidence: It Depends on How, What, and Whom You Ask. , 1999, Organizational behavior and human decision processes.

[18]  B. Brehmer,et al.  Chapter 3 What Have we Learned about Human Judgment from Thirty Years of Policy Capturing , 1988 .

[19]  J. Klayman.  Varieties of Confirmation Bias , 1995 .

[20]  William R. Ferrell,et al.  A model of calibration for subjective probabilities , 1980 .

[21]  R. Hogarth,et al.  Order effects in belief updating: The belief-adjustment model , 1992, Cognitive Psychology.

[22]  K. R. Hammond.  Human judgment and social policy , 1980 .

[23]  N. Epley,et al.  Putting adjustment back in the anchoring and adjustment heuristic. , 2002 .

[24]  T. R. Stewart.  Chapter 2 Judgment Analysis: Procedures , 1988 .

[25]  S. Oskamp.  OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[26]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[27]  Eric J. Johnson,et al.  Incorporating the Irrelevant: Anchors in Judgments of Belief and Value , 2002 .

[28]  Anders Winman,et al.  Information Sampling and Adaptive Cognition: Subjective Confidence and the Sampling of Knowledge , 2005 .

[29]  I. Erev,et al.  Simultaneous Over- and Underconfidence: The Role of Error in Judgment Processes. , 1994 .

[30]  D. Kahneman,et al.  Heuristics and Biases: The Psychology of Intuitive Judgment , 2002 .

[31]  Klaus Fiedler,et al.  Information Sampling and Adaptive Cognition , 2005 .

[32]  P. Juslin,et al.  Thurstonian and Brunswikian origins of uncertainty in judgment: a sampling model of confidence in sensory discrimination. , 1997, Psychological review.

[33]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[34]  Jack B. Soll.  Intuitive Theories of Information: Beliefs about the Value of Redundancy , 1999, Cognitive Psychology.

[35]  J. Frank Yates,et al.  Judgment and Decision Making , 1990 .

[36]  Paul Slovic,et al.  Behavioral problems of adhering to a decision policy , 1973 .

[37]  S. Lichtenstein,et al.  The Effects of Gender and Instructions on Calibration. , 1981 .

[38]  Richard P. Larrick,et al.  Motivational factors in decision theories : the role of self-protection , 1993 .

[39]  P. Todd,et al.  Simple Heuristics That Make Us Smart , 1999 .

[40]  R. Hastie,et al.  The relationship between memory and judgment depends on whether the judgment task is memory-based or on-line , 1986 .

[41]  Joshua Klayman,et al.  Overconfidence in interval estimates. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[42]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[43]  Dane K. Peterson,et al.  Confidence, uncertainty, and the use of information , 1988 .

[44]  E F Loftus,et al.  Trivial persuasion in the courtroom: the power of (a few) minor details. , 1989, Journal of personality and social psychology.

[45]  Margaret G. Meloy,et al.  The Distortion of Information during Decisions , 1996 .

[46]  Arthur C. Graesser,et al.  Narrative Comprehension, Causality, and Coherence : Essays in Honor of Tom Trabasso , 1999 .

[47]  P. Juslin,et al.  Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect. , 2000, Psychological review.

[48]  M. Lundeberg,et al.  Highly Confident, but Wrong: Gender Differences and Similarities in Confidence Judgments. , 1994 .

[49]  W. Ferrell,et al.  The Hard-Easy Effect in Subjective Probability Calibration , 1996 .

[50]  A. Tversky,et al.  Preference and belief: Ambiguity and competence in choice under uncertainty , 1991 .

[51]  Jack B. Soll.  Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure☆ , 1996 .

[52]  Nancy Pennington,et al.  Explanation-based decision making. , 2000 .

[53]  M. Kaplan,et al.  Human judgment and decision processes , 1975 .

[54]  William R. Ferrell,et al.  Discrete subjective probabilities and decision analysis: Elicitation, calibration and combination. , 1994 .

[55]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[56]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[57]  M. Degroot,et al.  Measuring utility by a single-response sequential method. , 1964, Behavioral science.

[58]  Thomas Mussweiler,et al.  Numeric Judgments under Uncertainty: The Role of Knowledge in Anchoring , 2000 .

[59]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[60]  Elisha Y. Babad,et al.  Wishful thinking and objectivity among sports fans. , 1987 .

[61]  A. D. Pearman,et al.  Judgment and Decision Making: An Interdisciplinary Reader , 1999 .

[62]  Fergus Bolger,et al.  The calibration of subjective probability: Theories and models 1980–94. , 1994 .

[63]  C. Gettys,et al.  MINERVA-DM: A memory processes model for judgments of likelihood. , 1999 .

[64]  Stephen J. Hoch,et al.  Counterfactual reasoning and accuracy in predicting personal events. , 1985 .

[65]  Ray W. Cooksey,et al.  Judgment analysis : theory, methods, and applications , 1996 .

[66]  D. Schum,et al.  Formal and empirical research on cascaded inference in jurisprudence. , 1982 .

[67]  Dane K. Peterson,et al.  Effects of amount of information on predictions of uncertain quantities , 1986 .

[68]  Peter Ayton,et al.  Organizational Behavior and Human Decision Processes Judgments of Decision Effectiveness: Actor–observer Differences in Overconfidence Straints. Recently, the Processes Underlying Judgmental and the Possibility That the Costs of Moving Further , 2022 .

[69]  T. R. Stewart,et al.  10 – SOCIAL JUDGMENT THEORY , 1975 .

[70]  Z. Kunda,et al.  The case for motivated reasoning. , 1990, Psychological bulletin.