Communicating forecasts: The simplicity of simulated experience

It is unclear whether decision makers who receive forecasts expressed as probability distributions over outcomes understand the implications of this form of communication. We suggest a solution based on the fact that people are effective at estimating the frequency of data accurately in environments that are characterized by plentiful, unbiased feedback. Thus, forecasters should provide decision makers with simulation models that allow them to experience the frequencies of potential outcomes. Before implementing this suggestion, however, it is important to assess whether people can make appropriate probabilistic inferences based on such simulated experience. In an experimental program, we find that statistically sophisticated and naive individuals relate easily to this presentation mode, they prefer it to analytic descriptions, and their probabilistic inferences improve. We conclude that asking decision makers to use simulations actively is potentially a powerful – and simplifying – method to improve the practice of forecasting.

[1]  Valerie F. Reyna,et al.  Neuroeconomics, judgment, and decision making , 2014 .

[2]  E. Yechiam,et al.  Consistent constructs in individuals' risk taking in decisions from experience. , 2010, Acta psychologica.

[3]  W. Sharpe,et al.  Choosing Outcomes Versus Choosing Products: Consumer-Focused Retirement Investment Advice , 2008 .

[4]  Emre Soyer,et al.  The Illusion of Predictability: How Regression Statistics Mislead Experts , 2011 .

[5]  L Hasher,et al.  Automatic processing of fundamental information: the case of frequency of occurrence. , 1984, The American psychologist.

[6]  Elliot A. Ludvig,et al.  Of Black Swans and Tossed Coins: Is the Description-Experience Gap in Risky Choice Limited to Rare Events? , 2011, PloS one.

[7]  Martin Weber,et al.  The Role of Experience Sampling and Graphical Displays on One's Investment Risk Appetite , 2012, Manag. Sci..

[8]  R. Hertwig,et al.  The description–experience gap in risky choice , 2009, Trends in Cognitive Sciences.

[9]  Spyros Makridakis,et al.  Forecasting and Planning: An Evaluation , 1981 .

[10]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[11]  R. Thaler,et al.  Risk Aversion Or Myopia? Choices in Repeated Gambles and Retirement Investments , 1999 .

[12]  B. Brehmer In one word: Not from experience. , 1980 .

[13]  J. Scott Armstrong,et al.  Principles of forecasting , 2001 .

[14]  R. Dawes,et al.  Heuristics and Biases: Clinical versus Actuarial Judgment , 2002 .

[15]  Nate Silver,et al.  The signal and the noise : why so many predictions fail but some don't , 2012 .

[16]  L. Hasher,et al.  Automatic and effortful processes in memory. , 1979 .

[17]  E. Yechiam,et al.  On the robustness of description and experience based decision tasks to social desirability , 2010 .

[18]  Lisa M. Schwartz,et al.  PSYCHOLOGICAL SCIENCE IN THE PUBLIC INTEREST Helping Doctors and Patients Make Sense of Health Statistics , 2022 .

[19]  Assessing the chances of success: naïve statistics versus kind experience. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[20]  J. Armstrong,et al.  The Seer-Sucker Theory: The Value of Experts in Forecasting , 2005 .

[21]  Emre Soyer,et al.  Sequentially simulated outcomes: kind experience versus nontransparent description. , 2011, Journal of experimental psychology. General.

[22]  R. Hertwig,et al.  Decisions from Experience and the Effect of Rare Events in Risky Choice , 2004, Psychological science.

[23]  Thomas T. Hills,et al.  Information Search in Decisions From Experience , 2010, Psychological science.

[24]  Lynn Hasher,et al.  Frequency processing: A twenty-five year perspective. , 2002 .

[25]  R. Hogarth Beyond discrete biases: Functional and dysfunctional aspects of judgmental heuristics. , 1981 .

[26]  Gerd Gigerenzer,et al.  How to Improve Bayesian Reasoning Without Instruction: Frequency Formats , 1995 .

[27]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[28]  Timothy J. Pleskac,et al.  Decisions from experience: Why small samples? , 2010, Cognition.

[29]  Kenneth R. Hammond,et al.  Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice , 2000 .

[30]  Varun Dutt,et al.  Instance-based learning: integrating sampling and repeated decisions from experience. , 2011, Psychological review.

[31]  Tilmann Betsch,et al.  A Sampling Approach to Biases in Conditional Probability Judgments: Beyond Baserate-Neglect and Statistical Format , 2000 .

[32]  E. Weber,et al.  Predicting Risk-Sensitivity in Humans and Lower Animals: Risk as Variance or Coefficient of Variation , 2004, Psychological review.

[33]  R. Thaler,et al.  Save More Tomorrow™: Using Behavioral Economics to Increase Employee Saving , 2004, Journal of Political Economy.

[34]  Cleotilde González,et al.  Effects of feedback and complexity on repeated decisions from description , 2011 .

[35]  S. Broomell,et al.  Effective communication of uncertainty in the IPCC reports , 2012, Climatic Change.

[36]  Gerd Gigerenzer,et al.  “A 30% Chance of Rain Tomorrow”: How Does the Public Understand Probabilistic Weather Forecasts? , 2005, Risk analysis : an official publication of the Society for Risk Analysis.

[37]  K. Fiedler Beware of samples! A cognitive-ecological sampling approach to judgment biases. , 2000, Psychological review.

[38]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[39]  Mark Buchanan Ubiquity: The Science of History . . . or Why the World Is Simpler Than We Think , 2000 .

[40]  Ben R. Newell,et al.  Degrees of uncertainty: An overview and framework for future research on experience‐based choice , 2010 .

[41]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[42]  N. Chater,et al.  Are Probabilities Overweighted or Underweighted When Rare Outcomes Are Experienced (Rarely)? , 2009, Psychological science.

[43]  D. Kahneman,et al.  Conditions for intuitive expertise: a failure to disagree. , 2009, The American psychologist.

[44]  S. Plous The psychology of judgment and decision making , 1994 .

[45]  R. Hogarth Intuition: A Challenge for Psychological Research on Decision Making , 2010 .

[46]  E. Hilgard,et al.  Theories of Learning , 1981 .