Probabilistic mental models: a Brunswikian theory of confidence.

Research on people's confidence in their general knowledge has to date produced two fairly stable effects, many inconsistent results, and no comprehensive theory. We propose such a comprehensive framework, the theory of probabilistic mental models (PMM theory). The theory (a) explains both the overconfidence effect (mean confidence is higher than percentage of answers correct) and the hard-easy effect (overconfidence increases with item difficulty) reported in the literature and (b) predicts conditions under which both effects appear, disappear, or invert. In addition, (c) it predicts a new phenomenon, the confidence-frequency effect, a systematic difference between a judgment of confidence in a single event (i.e., that any given answer is correct) and a judgment of the frequency of correct answers in the long run. Two experiments are reported that support PMM theory by confirming these predictions, and several apparent anomalies reported in the literature are explained and integrated into the present framework.

[1]  Helly.  Wahrscheinlichkeit, Statistik und Wahrheit , 1936 .

[2]  E. Brunswik.  Organismic achievement and environmental probability. , 1943 .

[3]  K. Duncker,et al.  On problem-solving , 1945 .

[4]  J. Swets,et al.  A decision-making theory of visual detection. , 1954, Psychological review.

[5]  K. Mather,et al.  Response to selection. , 1955, Cold Spring Harbor symposia on quantitative biology.

[6]  E. Brunswik.  Representative design and probabilistic theory in a functional psychology. , 1955, Psychological review.

[7]  Jerome S. Bruner,et al.  Contemporary approaches to cognition , 1958 .

[8]  A. Parducci.  Category judgment: a range-frequency model. , 1965, Psychological review.

[9]  Ian Hacking.  Logic of Statistical Inference , 1965 .

[10]  M. Scheerer,et al.  Problem Solving , 1967, Nature.

[11]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[12]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[13]  B. Armelius,et al.  The use of redundancy in multiple-cue judgments: Data from a suppressor-variable task. , 1974 .

[14]  Karl Halvor Teigen,et al.  Overestimation of subjective probabilities , 1974 .

[15]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[16]  W. Stegmüller.  Probleme und Resultate der Wissenschaftstheorie und analytischen Philosophie , 1974 .

[17]  M. Kaplan,et al.  Human judgment and decision processes , 1975 .

[18]  D. Allport.  The state of cognitive psychology. , 1975 .

[19]  Eugene Borgida,et al.  Attribution and the psychology of prediction. , 1975 .

[20]  T. R. Stewart,et al.  10 – SOCIAL JUDGMENT THEORY , 1975 .

[21]  R. D. Hansen,et al.  The power of consensus: Information derived from one's own and others' behavior. , 1977 .

[22]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[23]  Gary L. Wells,et al.  Do people use consensus information in making causal attributions , 1977 .

[24]  Thomas C. Toppino,et al.  Frequency and the Conference of Referential Validity. , 1977 .

[25]  Baruch Fischhoff,et al.  Calibration of Probabilities: The State of the Art , 1977 .

[26]  B. Fischhoff,et al.  Knowing with Certainty: The Appropriateness of Extreme Confidence. , 1977 .

[27]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[28]  G. Shafer.  Non-additive probabilities in the work of Bernoulli and Lambert , 1978, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[29]  Call for programs , 1978 .

[30]  W. Casscells,et al.  Interpretation by physicians of clinical laboratory results. , 1978, The New England journal of medicine.

[31]  W. C. Howell,et al.  Uncertainty measurement: A cognitive taxonomy , 1978 .

[32]  E. Rosch,et al.  Cognition and Categorization , 1980 .

[33]  K. Armelius,et al.  Task predictability and performance as determinants of confidence in multiple-cue judgments , 1979 .

[34]  L. Hasher,et al.  Automatic and effortful processes in memory. , 1979 .

[35]  D. M. Grether,et al.  Bayes Rule as a Descriptive Model: The Representativeness Heuristic , 1980 .

[36]  K. R. Hammond,et al.  Realizations of Brunswik's representative design , 1980 .

[37]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[38]  B. Fischhoff,et al.  Journal of Experimental Psychology: Human Learning and Memory , 1980 .

[39]  R. Hogarth,et al.  BEHAVIORAL DECISION THEORY: PROCESSES OF JUDGMENT AND CHOICE , 1981 .

[40]  D. Finlay,et al.  Perceived duration of auditory sequences. , 1982, The Journal of general psychology.

[41]  A. Tversky,et al.  On the study of statistical intuitions , 1982, Cognition.

[42]  Joseph B. Kadane,et al.  A Subjectivist View of Calibration. , 1982 .

[43]  Douglas L. Hintzman,et al.  Frequency as a nonpropositional attribute of memory. , 1982 .

[44]  B. Fischhoff,et al.  Subjective Confidence in Forecasts. , 1982 .

[45]  Sarah Lichtenstein,et al.  A Subjectivist View of Calibration (Report No. 82-6) , 1982 .

[46]  Automatic encoding of event frequency: Further findings. , 1982 .

[47]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[48]  Alf Zimmer,et al.  Verbal Vs. Numerical Processing of Subjective Probabilities , 1983 .

[49]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[50]  Peter H. Schönemann,et al.  Some theory and results for metrics for bounded response scales , 1983 .

[51]  Jeffrey E. Danes,et al.  Social Judgment Theory , 1984 .

[52]  Gerd Gigerenzer,et al.  External validity of laboratory experiments: The frequency-validity relationship , 1984 .

[53]  F. Lad.  The Calibration Question , 1984, The British Journal for the Philosophy of Science.

[54]  Willem A. Wagenaar,et al.  Calibration of probability assessments by professional blackjack dealers, statistical experts, and lay people , 1985 .

[55]  Alf C. Zimmer,et al.  What Uncertainty Judgments Can Tell About the Underlying Subjective Probabilities , 1985, UAI.

[56]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[57]  P Slovic,et al.  Perceived acceptability of risk analysis as a decision-making approach. , 1986, Risk analysis : an official publication of the Society for Risk Analysis.

[58]  R. May,et al.  Overconfidence as a result of incomplete and wrong knowledge. , 1986 .

[59]  Roland W. Scholz,et al.  Current issues in West German decision research , 1986 .

[60]  Berndt Brehmer,et al.  New directions in research on decision making , 1986 .

[61]  Arie W. Kruglanski,et al.  What Makes You So Sure? Effects of Epistemic Motivations on judgmental Confidence , 1987 .

[62]  M. Björkman.  A note on cue probability learning: what conditioning data reveal about cue contrast , 1987 .

[63]  Carl Martin Allwood,et al.  Response selection strategies and realism of confidence judgments , 1987 .

[64]  Yaacov Trope,et al.  Problem solving in judgment under uncertainty. , 1987 .

[65]  G. Gigerenzer,et al.  Cognition as Intuitive Statistics , 1987 .

[66]  D. E. Leary,et al.  From Act Psychology to Problematic Functionalism: The Place of Egon Brunswik in the History of Psychology , 1987 .

[67]  Gerd Gigerenzer,et al.  Survival of the fittest probabilist: Brunswik, Thurstone, and the two disciplines of psychology , 1987 .

[68]  Regine S. May.  Realismus von subjektiven Wahrscheinlichkeiten : eine kognitionspsychologische Analyse inferentieller Prozesse beim Overconfidence-Phänomen , 1987 .

[69]  Gideon Keren,et al.  Facing uncertainty in the game of bridge: A calibration study , 1987 .

[70]  David L. Ronis,et al.  Components of probability judgment accuracy: Individual consistency and effects of subject matter and assessment method. , 1987 .

[71]  William R. Woodward,et al.  PSYCHOLOGY IN TWENTIETH-CENTURY THOUGHT AND SOCIETY , 1991, Clinical Psychology Forum.

[72]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[73]  Gerd Gigerenzer,et al.  The probabilistic revolution: Vol. 2. Ideas in the sciences , 1987 .

[74]  Gerd Gigerenzer,et al.  The Probabilistic revolution , 1987 .

[75]  Yaacov Trope,et al.  Problem Solving in Judgment Under Uncertainty , 1987 .

[76]  A. D. Pearman,et al.  Judgment and Decision Making: An Interdisciplinary Reader , 1999 .

[77]  L. Daston.  Classical Probability in the Enlightenment , 1988 .

[78]  G. Keren,et al.  On the ability of monitoring non-veridical perceptions and uncertain knowledge: some calibration studies. , 1988, Acta psychologica.

[79]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[80]  B. Brehmer,et al.  Human judgment : the SJT view , 1988 .

[81]  B. Brehmer.  Chapter 1 The Development of Social Judgment Theory , 1988 .

[82]  K. Fiedler.  The dependence of the conjunction fallacy on subtle linguistic factors , 1988 .

[83]  Gerd Gigerenzer,et al.  Presentation and content: The use of base rates as a continuous variable. , 1988 .

[84]  John Beatty,et al.  The Empire of Chance: How Probability Changed Science and Everyday Life , 1989 .

[85]  L. N. Kanal,et al.  Uncertainty in Artificial Intelligence 5 , 1990 .

[86]  Gerd Gigerenzer,et al.  Context effects and their interaction with development: Area judgments , 1990 .

[87]  An Introduction to the Philosophy of Induction and Probability , 1990 .

[88]  G. Gigerenzer.  How to Make Cognitive Illusions Disappear: Beyond “Heuristics and Biases” , 1991 .

[89]  G. Gigerenzer.  From Tools to Theories: A Heuristic of Discovery in Cognitive Psychology. , 1991 .