The lure of incredible certitude

Abstract Forthright characterization of scientific uncertainty is important in principle and in practice. Nevertheless, economists and other researchers commonly report findings with incredible certitude, reporting point predictions and estimates. To motivate expression of incredible certitude, economists suggest that researchers respond to incentives that make the practice tempting. This temptation is the ‘lure’ of incredible certitude. I appraise some of the rationales that observers may have in mind when they state that incredible certitude responds to incentives. I conclude that scientific expression of incredible certitude at most has appeal in limited contexts. It should not be a general practice.

[1]  Charles F. Manski,et al.  Confidence Intervals for Partially Identified Parameters , 2003 .

[2]  Dan A. Black,et al.  Do Right‐To‐Carry Laws Deter Violent Crime? , 1998, The Journal of Legal Studies.

[3]  Charles E. Dagit,et al.  The Cost Estimate , 2017 .

[4]  Charles F. Manski,et al.  Learning about Treatment Effects from Experiments with Random Assignment of Treatments , 1996 .

[5]  D A Stainforth,et al.  Confidence, uncertainty and decision-support relevance in climate predictions , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[6]  On the Accuracy of Economic Observations. , 1951 .

[7]  Steven R. Lerman,et al.  The Estimation of Choice Probabilities from Choice Based Samples , 1977 .

[8]  Effects of Census Accuracy on Apportionment of Congress and Allocations of Federal Funds , 2015 .

[9]  A. Auerbach.  Dynamic Revenue Estimation , 1996 .

[10]  R. Kessler,et al.  Patterns and correlates of generalized anxiety disorder in community samples. , 2002, The Journal of clinical psychiatry.

[11]  Richard P. Brief.  The Accountant's Responsibility in Historical Perspective , 2014 .

[12]  Norman A. McFarlane,et al.  Parameterizations: representing key processes in climate models without resolving them , 2011 .

[13]  S. Durlauf,et al.  Model Uncertainty and the Effect of Shall-Issue Right-to-Carry Laws on Crime , 2015 .

[14]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[15]  Charles F. Manski,et al.  Censoring of Outcomes and Regressors Due to Survey Nonresponse: Identification and estimation Using Weights and Imputations , 1998 .

[16]  Angus Deaton,et al.  Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development , 2009 .

[17]  Giovanni Parmigiani,et al.  Meta-analysis of BRCA1 and BRCA2 penetrance. , 2007, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[18]  Markus K. Brunnermeier,et al.  Optimal Expectations , 2004 .

[19]  J. Cornfield,et al.  A method of estimating comparative rates from clinical data; applications to cancer of the lung, breast, and cervix. , 1951, Journal of the National Cancer Institute.

[20]  Timothy D. Wilson,et al.  The feeling of uncertainty intensifies affective reactions. , 2009, Emotion.

[21]  O. Morgenstern,et al.  On the Accuracy of Economic Observations. , 1950 .

[22]  James J Heckman,et al.  Comparing IV with Structural Models: What Simple IV Can and Cannot Identify , 2009, Journal of econometrics.

[23]  Mohd Hamid,et al.  Breast Cancer Risk Assessment Tool , 2014 .

[24]  Charles F. Manski,et al.  Estimation of Response Probabilities From Augmented Retrospective Observations , 1985 .

[25]  Charles F. Manski,et al.  How Do Right-to-Carry Laws Affect Crime Rates? Coping with Ambiguity Using Bounded-Variation Assumptions , 2015, Review of Economics and Statistics.

[26]  Robert F. Barsky.  Noam Chomsky: A Life of Dissent , 1997 .

[27]  C. Manski,et al.  Assessment of Two Cost-Effectiveness Studies on Cocaine Control Policy , 1999 .

[28]  Elizabeth C. Hirschman,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[29]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[30]  J. Lott.  More Guns, Less Crime: Understanding Crime and Gun-Control Laws , 1998 .

[31]  Susan Hoover,et al.  Breast cancer screening and diagnosis. , 2006, Journal of the National Comprehensive Cancer Network : JNCCN.

[32]  C. Manski,et al.  Committee On Data And Research For Policy On Illegal Drugs , 1999 .

[33]  Charles F. Manski,et al.  Communicating uncertainty in policy analysis , 2018, Proceedings of the National Academy of Sciences.

[34]  Charles F. Manski,et al.  The Selection Problem , 1990 .

[35]  J. Angrist,et al.  Identification and Estimation of Local Average Treatment Effects , 1995 .

[36]  Original Pronouncements,et al.  Statement of Financial Accounting Concepts No . 7 Using Cash Flow Information and Present Value in Accounting Measurements Financial Accounting Standards Board , 2003 .

[37]  M. Dugas,et al.  Cognitive-Behavioral Treatment for Generalized Anxiety Disorder: From Science to Practice , 2006 .

[38]  G. Gigerenzer,et al.  Probabilistic mental models: a Brunswikian theory of confidence. , 1991, Psychological review.

[39]  Wendy S. Parker,et al.  Understanding Pluralism in Climate Modeling , 2006 .

[40]  C. Manski.  Partial Identification of Probability Distributions , 2003 .

[41]  George A. Akerlof,et al.  The Economic Consequences of Cognitive Dissonance , 1982 .

[42]  C. Manski.  Reasonable patient care under uncertainty. , 2018, Health Economics.

[43]  Henry Buchwald,et al.  Bariatric surgery: a systematic review and meta-analysis. , 2004, JAMA.

[44]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[45]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[46]  J. Fleiss.  Statistical methods for rates and proportions , 1974 .

[47]  David B. Mustard,et al.  Crime, Deterrence, and Right‐to‐Carry Concealed Handguns , 1997, The Journal of Legal Studies.

[48]  B. Fischhoff,et al.  Subjective Confidence in Forecasts. , 1982 .

[49]  Francis W. Zwiers,et al.  Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties , 2010 .

[50]  N. Laird,et al.  Meta-analysis in clinical trials revisited. , 2015, Contemporary clinical trials.

[51]  M. Bonazountas,et al.  Risk assessment : supporting public policy in an uncertain world , 2017 .

[52]  P. Muris,et al.  Randomized controlled trial on the effectiveness of metacognitive therapy and intolerance-of-uncertainty therapy for generalized anxiety disorder. , 2012, Behaviour research and therapy.

[53]  David Kjellberg.  Measuring Expectations , 1999, Identification Problems in the Social Sciences.

[54]  Joshua D. Angrist,et al.  Identification of Causal Effects Using Instrumental Variables , 1993 .

[55]  Timothy D. Wilson,et al.  The pleasures of uncertainty: prolonging positive moods in ways people do not anticipate. , 2005, Journal of personality and social psychology.

[56]  M. Gail,et al.  Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. , 1989, Journal of the National Cancer Institute.

[57]  Murray B Stein,et al.  Anxiety , 2017, The Lancet.

[58]  Wendy S. Parker,et al.  Ensemble modeling, uncertainty and robust predictions , 2013 .

[59]  M. Duggan,et al.  More Guns, More Crime , 2000, Journal of Political Economy.

[60]  M. Hurd.  Subjective Probabilities in Household Surveys. , 2009, Annual review of economics.

[61]  C. Manski.  Survey Measurement of Probabilistic Macroeconomic Expectations: Progress and Promise , 2017, NBER Macroeconomics Annual.

[62]  Reto Knutti,et al.  Challenges in Combining Projections from Multiple Climate Models , 2010 .

[63]  Renate Hagedorn,et al.  Representing model uncertainty in weather and climate prediction , 2005 .

[64]  Simon M. Potter,et al.  Measuring Inflation Expectations , 2013 .

[65]  J. Angrist,et al.  Identification and Estimation of Local Average Treatment Effects , 1994 .

[66]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[67]  Z. Kunda,et al.  The case for motivated reasoning. , 1990, Psychological bulletin.

[68]  Charles F. Manski.  The Lure of Incredible Certitude , 2018 .

[69]  Charles F. Manski,et al.  DIVERSIFIED TREATMENT UNDER AMBIGUITY , 2008 .

[70]  James C McWilliams,et al.  Irreducible imprecision in atmospheric and oceanic simulations , 2007, Proceedings of the National Academy of Sciences.

[71]  C. Manski,et al.  The 2009 Lawrence R. Klein Lecture: Diversified Treatment Under Ambiguity , 2009 .

[72]  K. Buhr,et al.  The role of fear of anxiety and intolerance of uncertainty in worry: an experimental manipulation. , 2009, Behaviour research and therapy.

[73]  Alexander L. Davis,et al.  Communicating scientific uncertainty , 2014, Proceedings of the National Academy of Sciences.

[74]  J. Lott.  More Guns, Less Crime: Understanding Crime and Gun Control Laws, Third Edition , 2010 .

[75]  Ann Henderson-Sellers,et al.  A Climate Modelling Primer: McGuffie/A Climate Modelling Primer , 2005 .

[76]  Joseph Berkson,et al.  Smoking and Lung Cancer: Some Observations on Two Recent Reports , 1958 .

[77]  P. Rosenbaum.  Choice as an Alternative to Control in Observational Studies , 1999 .

[78]  C. Manski.  Anatomy of the Selection Problem , 1989 .

[79]  Paul H. Richanbach,et al.  An Empirical Examination of Counterdrug Interdiction Program Effectiveness. , 1997 .

[80]  C. Manski.  Policy Analysis with Incredible Certitude , 2010 .

[81]  M. Sobel,et al.  Identification Problems in the Social Sciences. , 1996 .

[82]  J. Tukey.  The Future of Data Analysis , 1962 .

[83]  Dean D. Croushore,et al.  Frontiers of Real-Time Data Analysis , 2008 .

[84]  Temple F. Smith.  Occam's razor , 1980, Nature.

[85]  Jean Tirole,et al.  Mindful Economics: The Production, Consumption, and Value of Beliefs , 2016 .

[86]  G. Imbens,et al.  Better Late than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009) , 2009 .

[87]  S. Kapadia,et al.  Uncertainty in macroeconomic policy-making: art or science? , 2011, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[88]  Charles F. Manski,et al.  Credible interval estimates for official statistics with survey nonresponse , 2016 .

[89]  Dennis Fixler,et al.  Revisions to GDP, GDI, and Their Major Components , 2011 .

[90]  Theodore A. Wells.  Mindful Economics : The Production , Consumption , and Value of Beliefs , 2016 .

[91]  N. Aplin,et al.  PSYCHOLOGICAL EXPECTED UTILITY THEORY AND ANTICIPATORY FEELINGS * A , 1997 .

[92]  Petra Kaufmann,et al.  Experimental And Quasi Experimental Designs For Research , 2016 .

[93]  C. Manski.  Communicating Uncertainty in Official Economic Statistics: An Appraisal Fifty Years after Morgenstern , 2015 .

[94]  David Hemenway,et al.  Firearms and violence: a critical review , 2006, Injury Prevention.

[95]  Christian Gollier,et al.  Optimal Choice and Beliefs with Ex Ante Savoring and Ex Post Disappointment , 2010, Manag. Sci..

[96]  Baruch Fischhoff,et al.  Communicating uncertainty: Fulfilling the duty to inform , 2012 .

[97]  A. Schotter,et al.  Belief Elicitation in the Laboratory , 2014 .

[98]  Charles F. Manski,et al.  Identification for Prediction and Decision , 2008 .

[99]  K. Buhr,et al.  The Intolerance of Uncertainty Scale: psychometric properties of the English version. , 2002, Behaviour research and therapy.

[100]  Andrew Caplin,et al.  Psychological Expected Utility Theory and Anticipatory Feelings , 2001 .

[101]  M. Friedman.  Essays in Positive Economics , 1954 .

[102]  Adeline Delavande.  Probabilistic Expectations in Developing Countries , 2014 .

[103]  “He Loves Me, He Loves Me Not . . . ” , 2011, Psychological science.