Communicating uncertainty in policy analysis

The term “policy analysis” describes scientific evaluations of the impacts of past public policies and predictions of the outcomes of potential future policies. A prevalent practice has been to report policy analysis with incredible certitude. That is, exact predictions of policy outcomes are routine, while expressions of uncertainty are rare. However, predictions and estimates often are fragile, resting on unsupported assumptions and limited data. Therefore, the expressed certitude is not credible. This paper summarizes my work documenting incredible certitude and calling for transparent communication of uncertainty. I present a typology of practices that contribute to incredible certitude, give illustrative examples, and offer suggestions on how to communicate uncertainty.

[1]  S. Kuznets.  National Income: A New Version , 1948 .

[2]  Dennis Fixler,et al.  Revisions to GDP, GDI, and Their Major Components , 2011 .

[3]  O. Morgenstern,et al.  On the Accuracy of Economic Observations. , 1950 .

[4]  C. Manski.  Policy Analysis with Incredible Certitude , 2010 .

[5]  次郎 永井.  Heritability , 1958, Current Biology.

[6]  Daniel S. Nagin,et al.  Deterrence and the Death Penalty , 2012 .

[7]  A. Blumstein,et al.  Deterrence and incapacitation : estimating the effects of criminal sanctions on crime rates , 1980 .

[8]  O. Morgenstern,et al.  On the Accuracy of Economic Observations. , 1950 .

[9]  C. Manski.  Communicating Uncertainty in Official Economic Statistics: An Appraisal Fifty Years after Morgenstern , 2015 .

[10]  David Hemenway,et al.  Firearms and violence: a critical review , 2006, Injury Prevention.

[11]  Charles F. Manski,et al.  Confidence Intervals for Partially Identified Parameters , 2003 .

[12]  Charles F. Manski,et al.  Credible interval estimates for official statistics with survey nonresponse , 2016 .

[13]  Baruch Fischhoff,et al.  Communicating uncertainty: Fulfilling the duty to inform , 2012 .

[14]  Dean D. Croushore,et al.  Frontiers of Real-Time Data Analysis , 2008 .

[15]  Alexander L. Davis,et al.  Communicating scientific uncertainty , 2014, Proceedings of the National Academy of Sciences.

[16]  S. Kapadia,et al.  Uncertainty in macroeconomic policy-making: art or science? , 2011, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[17]  Joseph R. Cummins.  The Role of Government in Education , 2018 .

[18]  Charles F. Manski,et al.  Deterrence and the Death Penalty: Partial Identification Analysis Using Repeated Cross Sections , 2011, Journal of Quantitative Criminology.

[19]  Constance F. Citro,et al.  Principles and practices for a federal statistical agency , 2005 .

[20]  Charles F. Manski,et al.  Censoring of Outcomes and Regressors Due to Survey Nonresponse: Identification and estimation Using Weights and Imputations , 1998 .

[21]  C. Manski.  Anatomy of the Selection Problem , 1989 .

[22]  Jonathan H. Wright.  Unseasonal Seasonals? , 2014 .

[23]  Ernest Van Den Hagg,et al.  On Deterrence and the Death Penalty , 1969 .

[24]  On the Accuracy of Economic Observations. , 1951 .

[25]  Charles F. Manski,et al.  The Selection Problem , 1990 .

[26]  C. Manski.  Partial Identification of Probability Distributions , 2003 .

[27]  M. Bonazountas,et al.  Risk assessment : supporting public policy in an uncertain world , 2017 .

[28]  Charles F. Manski,et al.  How Do Right-to-Carry Laws Affect Crime Rates? Coping with Ambiguity Using Bounded-Variation Assumptions , 2015, Review of Economics and Statistics.

[29]  C. Manski,et al.  The 2009 Lawrence R. Klein Lecture: Diversified Treatment Under Ambiguity , 2009 .

[30]  Charles F. Manski,et al.  Identification for Prediction and Decision , 2008 .

[31]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[32]  Charles F. Manski,et al.  DIVERSIFIED TREATMENT UNDER AMBIGUITY , 2008 .