Expert judgement and uncertainty quantification for climate change

Expert judgement is often used to assess uncertainties in model-based climate change projections. This Perspective describes a statistical approach to formalizing the role of expert judgement, using Antarctic ice loss as an illustrative example. Expert judgement is an unavoidable element of the process-based numerical models used for climate change projections, and the statistical approaches used to characterize uncertainty across model ensembles. Here, we highlight the need for formalized approaches to unifying numerical modelling with expert judgement in order to facilitate characterization of uncertainty in a reproducible, consistent and transparent fashion. As an example, we use probabilistic inversion, a well-established technique used in many other applications outside of climate change, to fuse two recent analyses of twenty-first century Antarctic ice loss. Probabilistic inversion is but one of many possible approaches to formalizing the role of expert judgement, and the Antarctic ice sheet is only one possible climate-related application. We recommend indicators or signposts that characterize successful science-based uncertainty quantification.

[1]  Imre Lakatos,et al.  The methodology of scientific research programmes: Contents , 1978 .

[2]  Naomi Oreskes,et al.  The rapid disintegration of projections: The West Antarctic Ice Sheet and the Intergovernmental Panel on Climate Change , 2012, Social studies of science.

[3]  Brian C. O’Neill,et al.  Negative learning , 2008 .

[4]  W. Deming,et al.  On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known , 1940 .

[5]  W. Aspinall A route to more tractable expert advice , 2010, Nature.

[6]  Matthew D. Collins,et al.  UK Climate Projections Science Report: Climate Change Projections , 2009 .

[7]  Katharine Hayhoe,et al.  Suburban watershed nitrogen retention: estimating the effectiveness of stormwater management structures , 2015 .

[8]  Susie ElSaadany,et al.  Expert Elicitation for the Judgment of Prion Disease Risk Uncertainties , 2011, Journal of toxicology and environmental health. Part A.

[9]  Gaël Durand,et al.  Reducing uncertainties in projections of Antarctic ice mass loss , 2015 .

[10]  淳一 清水,et al.  学界情報:13th International Conference on Greenhouse Gas Control Technologies 報告 , 2017 .

[11]  David A. Rhoades,et al.  Seismic Hazard Modeling for the Recovery of Christchurch , 2014 .

[12]  Roger M. Cooke,et al.  MORTALITY IN KUWAIT DUE TO PM FROM OIL FIRES AFTER THE GULF WAR: COMBINING EXPERT ELICITATION ASSESSMENTS , 2005 .

[13]  J. Palutikof,et al.  Climate change 2007 : impacts, adaptation and vulnerability , 2001 .

[14]  Robert S. Anderson,et al.  Risk and Uncertainty Assessment for Natural Hazards , 2014 .

[15]  Gaël Durand,et al.  Grounding-line migration in plan-view marine ice-sheet models: results of the ice2sea MISMIP3d intercomparison , 2013, Journal of Glaciology.

[16]  Tine Hald,et al.  Research Synthesis Methods in an Age of Globalized Risks: Lessons from the Global Burden of Foodborne Disease Expert Elicitation , 2016, Risk analysis : an official publication of the Society for Risk Analysis.

[17]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[18]  M Granger Morgan,et al.  Expert judgments about transient climate response to alternative future trajectories of radiative forcing , 2009, Proceedings of the National Academy of Sciences.

[19]  R. Sparks,et al.  Risk and Uncertainty Assessment for Natural Hazards , 2013 .

[20]  Chakib Bouallou,et al.  Study of an innovative gas-liquid contactor for CO2 absorption , 2011 .

[21]  Eric Rignot,et al.  Insights into spatial sensitivities of ice mass response to environmental change from the SeaRISE ice sheet modeling project II: Greenland , 2013, Journal of Geophysical Research: Earth Surface.

[22]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[23]  Simon Shackley,et al.  Anchoring Devices in Science for Policy , 1998 .

[24]  K. Popper,et al.  The Logic of Scientific Discovery , 1960 .

[25]  Willy P Aspinall,et al.  An expert judgement assessment of future sea level rise from the ice sheets , 2013 .

[26]  Leonard A. Smith,et al.  The Myopia of Imperfect Climate Models: The Case of UKCP09 , 2013, Philosophy of Science.

[27]  Charles B. Connor,et al.  Statistics in Volcanology , 2006 .

[28]  Roger M Cooke,et al.  Out-of-sample validation for structured expert judgment of Asian carp establishment in Lake Erie , 2014, Integrated environmental assessment and management.

[29]  Tim Bedford,et al.  Probabilistic Inversion of Expert Judgments in the Quantification of Model Uncertainty , 2005, Manag. Sci..

[30]  M. Saier,et al.  Climate Change, 2007 , 2007 .

[31]  Roger M. Cooke,et al.  Quantifying scientific uncertainty from expert judgement elicitation , 2013 .

[32]  Marko Tainio,et al.  A probabilistic characterization of the relationship between fine particulate matter and mortality: elicitation of European experts. , 2007, Environmental science & technology.

[33]  Richard B. Alley,et al.  Potential Antarctic Ice Sheet retreat driven by hydrofracturing and ice cliff failure , 2015 .

[34]  Mitchell J. Small,et al.  Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Decision Making , 2009 .

[35]  Roger M. Cooke,et al.  Techniques for generic probabilistic inversion , 2006, Comput. Stat. Data Anal..

[36]  William H. Lipscomb,et al.  Ice-sheet model sensitivities to environmental forcing and their use in projecting future sea level (the SeaRISE project) , 2013, Journal of Glaciology.

[37]  Michael Oppenheimer,et al.  Probabilistic framework for assessing the ice sheet contribution to sea level change , 2013, Proceedings of the National Academy of Sciences.

[38]  Brian C. O'Neill,et al.  The Limits of Consensus , 2007, Science.

[39]  J. Clawson,et al.  Greenland , 2010, Nature.

[40]  Michael Goldstein,et al.  Reified Bayesian modelling and inference for physical systems , 2009 .

[41]  Reto Knutti,et al.  Challenges in Combining Projections from Multiple Climate Models , 2010 .

[42]  I. Csiszár $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[43]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[44]  T. Stocker,et al.  SBSTA-IPCC Special Event Climate Change 2013: The Physical Science Basis , 2013 .

[45]  Reto Knutti,et al.  The equilibrium sensitivity of the Earth's temperature to radiation changes , 2008 .

[46]  Roger M. Cooke,et al.  Using Structured Expert Judgment to Assess Invasive Species Prevention: Asian Carp and the Mississippi—Great Lakes Hydrologic Connection , 2014, Environmental science & technology.

[47]  W. H. Farland,et al.  Workshop report on EPA (Environmental Protection Agency) guidelines for carcinogen risk assessment. Held in Virginia Beach, Virginia on January 11-13, 1989 , 1989 .

[48]  Roderik S. W. van de Wal,et al.  How to interpret expert judgment assessments of 21st century sea-level rise , 2015 .

[49]  M. G. Morgan,et al.  Subjective judgments by climate experts. , 1995 .

[50]  M. Webb,et al.  Multivariate probabilistic projections using imperfect climate models part I: outline of methodology , 2012, Climate Dynamics.

[51]  Reto Knutti,et al.  The use of the multi-model ensemble in probabilistic climate projections , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[52]  David M. H. Sexton,et al.  Multivariate probabilistic projections using imperfect climate models. Part II: robustness of methodological choices and consequences for climate sensitivity , 2012, Climate Dynamics.

[53]  Nathan M. Urban,et al.  Upper bounds on twenty-first-century Antarctic ice loss assessed using a probabilistic framework , 2013 .

[54]  R. Knutti,et al.  Predictor Screening, Calibration, and Observational Constraints in Climate Model Ensembles: An Illustration Using Climate Sensitivity , 2013 .

[55]  Michael Goldstein,et al.  Second-Order Exchangeability Analysis for Multimodel Ensembles , 2013 .

[56]  Roger M Cooke,et al.  Use of structured expert judgment to forecast invasions by bighead and silver carp in Lake Erie , 2015, Conservation biology : the journal of the Society for Conservation Biology.

[57]  B. Smith,et al.  Marine Ice Sheet Collapse Potentially Under Way for the Thwaites Glacier Basin, West Antarctica , 2014, Science.

[58]  J. Murphy,et al.  A methodology for probabilistic predictions of regional climate change from perturbed physics ensembles , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[59]  Roger M. Cooke,et al.  Uncertainty in dispersion and deposition in accident consequence modeling assessed with performance-based expert judgment , 1994 .

[60]  Aslak Grinsted,et al.  Upper limit for sea level projections by 2100 , 2014 .

[61]  Roger M. Cooke,et al.  WHAT RISK ASSESSMENT CAN TELL US ABOUT THE MORTALITY IMPACTS OF THE KUWAITI OIL FIRES , 2005 .

[62]  Francis W. Zwiers,et al.  Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties , 2010 .

[63]  Roger M. Cooke,et al.  TU Delft expert judgment data base , 2008, Reliab. Eng. Syst. Saf..

[64]  Corinne Le Quéré,et al.  Climate Change 2013: The Physical Science Basis , 2013 .

[65]  Tom Addiscott,et al.  Science and Judgement in Risk Assessment , 1997 .

[66]  Martin Vermeer,et al.  Testing the robustness of semi-empirical sea level projections , 2012, Climate Dynamics.

[67]  Shahram Sarkani,et al.  The effect of the number of seed variables on the performance of Cooke′s classical model , 2014, Reliab. Eng. Syst. Saf..

[68]  Wendy S. Parker,et al.  Ensemble modeling, uncertainty and robust predictions , 2013 .

[69]  B. Scheuchl,et al.  Widespread, rapid grounding line retreat of Pine Island, Thwaites, Smith, and Kohler glaciers, West Antarctica, from 1992 to 2011 , 2014 .

[70]  C. Tebaldi,et al.  Probabilistic 21st and 22nd century sea‐level projections at a global network of tide‐gauge sites , 2014 .

[71]  John A. Hall,et al.  Global sea level rise scenarios for the United States National Climate Assessment , 2012 .

[72]  David G. Vaughan,et al.  Risk Estimation of Collapse of the West Antarctic Ice Sheet , 2002 .

[73]  X. Fettweis,et al.  Coastal flood damage and adaptation costs under 21st century sea-level rise , 2014, Proceedings of the National Academy of Sciences.

[74]  Benjamin P. Horton,et al.  Expert assessment of sea-level rise by AD 2100 and AD 2300 , 2014 .

[75]  Imre Lakatos,et al.  The Methodology of Scientific Research Programmes , 1978 .