Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling

Critical decisions frequently rely on high-dimensional output from complex computer simulation models that show intricate cross-variable, spatial and temporal dependence structures, with weather and climate predictions being key examples. There is a strongly increasing recognition of the need for uncertainty quantification in such settings, for which we propose and review a general multi-stage procedure called ensemble copula coupling (ECC), proceeding as follows: 1. Generate a raw ensemble, consisting of multiple runs of the computer model that differ in the inputs or model parameters in suitable ways. 2. Apply statistical postprocessing techniques, such as Bayesian model averaging or nonhomogeneous regression, to correct for systematic errors in the raw ensemble, to obtain calibrated and sharp predictive distributions for each univariate output variable individually. 3. Draw a sample from each postprocessed predictive distribution. 4. Rearrange the sampled values in the rank order structure of the raw ensemble to obtain the ECC postprocessed ensemble. The use of ensembles and statistical postprocessing have become routine in weather forecasting over the past decade. We show that seemingly unrelated, recent advances can be interpreted, fused and consolidated within the framework of ECC, the common thread being the adoption of the empirical copula of the raw ensemble. Depending on the use of Quantiles, Random draws or Transformations at the sampling stage, we distinguish the ECC-Q, ECC-R and ECC-T variants, respectively. We also describe relations to the Schaake shuffle and extant copula-based techniques. In a case study, the ECC approach is applied to predictions of temperature, pressure, precipitation and wind over Germany, based on the 50-member European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble.

[1]  Ernest Cooke,et al.  FORECASTS AND VERIFICATIONS IN WESTERN AUSTRALIA , 2022 .

[2]  M. Sklar Fonctions de repartition a n dimensions et leurs marges , 1959 .

[3]  W. Hays Statistical theory. , 1968, Annual review of psychology.

[4]  Ludger Rüschendorf,et al.  Asymptotic Distributions of Multivariate Rank Order Statistics , 1976 .

[5]  Winfried Stute,et al.  The Oscillation Behavior of Empirical Processes: The Multivariate Case , 1984 .

[6]  Roman Krzysztofowicz,et al.  Probabilistic Quantitative Precipitation Forecasts for River Basins , 1993, Weather and Forecasting.

[7]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[8]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[9]  Thomas M. Hamill,et al.  Verification of Eta–RSM Short-Range Ensemble Forecasts , 1997 .

[10]  H. Joe Multivariate models and dependence concepts , 1998 .

[11]  R. Nelsen An Introduction to Copulas , 1998 .

[12]  J. Chilès,et al.  Geostatistics: Modeling Spatial Uncertainty , 1999 .

[13]  S. Graf,et al.  Foundations of Quantization for Probability Distributions , 2000 .

[14]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[15]  T. Hamill Interpretation of Rank Histograms for Verifying Ensemble Forecasts , 2001 .

[16]  Tim N. Palmer,et al.  The economic value of ensemble forecasts as a tool for risk assessment: From days to decades , 2002 .

[17]  Eugenia Kalnay,et al.  Atmospheric Modeling, Data Assimilation and Predictability , 2002 .

[18]  Daniel S. Wilks,et al.  Smoothing forecast ensembles with fitted probability distributions , 2002 .

[19]  Tilmann Gneiting,et al.  Calibrated Probabilistic Mesoscale Weather Field Forecasting , 2004 .

[20]  M. Wegkamp,et al.  Weak Convergence of Empirical Copula Processes , 2004 .

[21]  John Bjørnar Bremnes,et al.  Probabilistic Forecasts of Precipitation in Terms of Quantiles Using NWP Model Output , 2004 .

[22]  M. Clark,et al.  The Schaake Shuffle: A Method for Reconstructing Space–Time Variability in Forecasted Precipitation and Temperature Fields , 2004 .

[23]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[24]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[25]  Adrian E. Raftery,et al.  Weather Forecasting with Ensemble Methods , 2005, Science.

[26]  Clifford F. Mass,et al.  Aspects of Effective Mesoscale, Short-Range Ensemble Forecasting , 2005 .

[27]  Christopher K. Wikle,et al.  Atmospheric Modeling, Data Assimilation, and Predictability , 2005, Technometrics.

[28]  R. Stull,et al.  Probabilistic aspects of meteorological and ozone regional ensemble forecasts , 2006 .

[29]  Christine M. Anderson-Cook,et al.  Book review: quantitative risk management: concepts, techniques and tools, revised edition, by A.F. McNeil, R. Frey and P. Embrechts. Princeton University Press, 2015, ISBN 978-0-691-16627-8, xix + 700 pp. , 2017, Extremes.

[30]  Radko Mesiar,et al.  Discrete Copulas , 2006, IEEE Transactions on Fuzzy Systems.

[31]  Mats Hamrud,et al.  The new ECMWF VAREPS (Variable Resolution Ensemble Prediction System) , 2007 .

[32]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[33]  Holger Dette,et al.  Non‐crossing non‐parametric estimates of quantile curves , 2008 .

[34]  Thomas M. Hamill,et al.  Comparison of Ensemble-MOS Methods Using GFS Reforecasts , 2007 .

[35]  Joan Torrens,et al.  Sklar's Theorem in Finite Settings , 2007, IEEE Transactions on Fuzzy Systems.

[36]  J. M. Sloughter,et al.  Probabilistic Quantitative Precipitation Forecasting Using Bayesian Model Averaging , 2007 .

[37]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[38]  C. Genest,et al.  Everything You Always Wanted to Know about Copula Modeling but Were Afraid to Ask , 2007 .

[39]  Adrian E. Raftery,et al.  Combining Spatial Statistical and Ensemble Information in Probabilistic Weather Forecasts , 2007 .

[40]  John Bjørnar Bremnes Improved calibration of precipitation forecasts using ensemble techniques Part 2 : Statistical calibration methods , 2007 .

[41]  A. McNeil,et al.  The t Copula and Related Copulas , 2005 .

[42]  J. Schaake,et al.  Precipitation and temperature ensemble forecasts from single-value forecasts , 2007 .

[43]  J. Lygeros,et al.  Effect of Wind Correlation on Aircraft Conflict Probability , 2007 .

[44]  Renate Hagedorn,et al.  Probabilistic Forecast Calibration Using ECMWF and GFS Ensemble Reforecasts. Part II: Precipitation , 2008 .

[45]  Stevie Roquelaure,et al.  A Local Ensemble Prediction System for Fog and Low Clouds: Construction, Bayesian Model Averaging Calibration, and Validation , 2008 .

[46]  Tilmann Gneiting,et al.  Editorial: Probabilistic forecasting , 2008 .

[47]  P. Friederichs,et al.  Multivariate non-normally distributed random variables in climate research - introduction to the copula approach , 2008 .

[48]  Tim N. Palmer,et al.  Ensemble forecasting , 2008, J. Comput. Phys..

[49]  L. Held,et al.  Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds , 2008 .

[50]  Renate Hagedorn,et al.  Probabilistic Forecast Calibration Using ECMWF and GFS Ensemble Reforecasts. Part I: Two-Meter Temperatures , 2008 .

[51]  Daniela Cocchi,et al.  A Bayesian Hierarchical Approach to Ensemble Weather Forecasting , 2008 .

[52]  Bob Glahn,et al.  Reforecasts: An Important Dataset for Improving Weather Predictions , 2008 .

[53]  Adrian E. Raftery,et al.  Probabilistic quantitative precipitation field forecasting using a two-stage spatial model , 2008, 0901.3484.

[54]  Leonard A. Smith,et al.  From ensemble forecasts to predictive distribution functions , 2008 .

[55]  T. Gneiting Making and Evaluating Point Forecasts , 2009, 0912.0902.

[56]  Daniel S. Wilks,et al.  Extending logistic regression to provide full‐probability‐distribution MOS forecasts , 2009 .

[57]  H. Madsen,et al.  From probabilistic forecasts to statistical scenarios of short-term wind power production , 2009 .

[58]  Claudia Czado,et al.  Predictive Model Assessment for Count Data , 2009, Biometrics.

[59]  A. Frigessi,et al.  Pair-copula constructions of multiple dependence , 2009 .

[60]  L. Rüschendorf On the distributional transform, Sklar's theorem, and the empirical copula process , 2009 .

[61]  Bob Glahn,et al.  MOS Uncertainty Estimates in an Ensemble Framework , 2009 .

[62]  Alexander J. McNeil,et al.  Multivariate Archimedean copulas, $d$-monotone functions and $\ell_1$-norm symmetric distributions , 2009, 0908.3750.

[63]  A. Kann,et al.  Calibrating 2-m Temperature of Limited-Area Ensemble Forecasts Using High-Resolution Analysis , 2009 .

[64]  Florian Pappenberger,et al.  Ensemble flood forecasting: a review. , 2009 .

[65]  Adrian E. Raftery,et al.  Prediction under Model Uncertainty Via Dynamic Model Averaging : Application to a Cold Rolling Mill 1 , 2008 .

[66]  A. Hense,et al.  Probabilistic assessment of regional climate change in Southwest Germany by ensemble dressing , 2011 .

[67]  Tilmann Gneiting,et al.  Probabilistic forecasts of wind speed: ensemble model output statistics by using heteroscedastic censored regression , 2010 .

[68]  M. Schmeits,et al.  A Comparison between Raw Ensemble Output, (Modified) Bayesian Model Averaging, and Extended Logistic Regression Using ECMWF Ensemble Precipitation Reforecasts , 2010 .

[69]  T. Hamill,et al.  Summary of recommendations of the first workshop on Postprocessing and Downscaling Atmospheric Forecasts for Hydrologic Applications held at Météo‐France, Toulouse, France, 15–18 June 2009 , 2010 .

[70]  Adrian E. Raftery,et al.  Bias Correction and Bayesian Model Averaging for Ensemble Forecasts of Surface Wind Direction , 2010 .

[71]  A. Raftery,et al.  Calibrating Multimodel Forecast Ensembles with Exchangeable and Missing Members Using Bayesian Model Averaging , 2010 .

[72]  A. Raftery,et al.  Probabilistic Weather Forecasting for Winter Road Maintenance , 2010 .

[73]  Adrian E. Raftery,et al.  Locally Calibrated Probabilistic Temperature Forecasting Using Geostatistical Model Averaging and Local Bayesian Model Averaging , 2011 .

[74]  Adrian E. Raftery,et al.  Probabilistic Visibility Forecasting Using Bayesian Model Averaging , 2011 .

[75]  Tilmann Gneiting,et al.  Geostatistical Model Averaging for Locally Calibrated Probabilistic Quantitative Precipitation Forecasting , 2011 .

[76]  Adrian E. Raftery,et al.  Probabilistic Weather Forecasting in R , 2011 .

[77]  Noel A Cressie,et al.  Statistics for Spatio-Temporal Data , 2011 .

[78]  Florian Pappenberger,et al.  Application of a medium-range global hydrologic probabilistic forecast scheme to the Ohio River Basin , 2011 .

[79]  Renate Hagedorn,et al.  Comparing TIGGE multimodel forecasts with reforecast‐calibrated ECMWF ensemble forecasts , 2012 .

[80]  T. Gneiting,et al.  Ensemble Model Output Statistics for Wind Vectors , 2012, 1201.2612.

[81]  Pierre Pinson,et al.  Adaptive calibration of (u,v)‐wind ensemble forecasts , 2012 .

[82]  A. Davison,et al.  Statistical Modeling of Spatial Extremes , 2012, 1208.3378.

[83]  J. Flowerdew Calibration and combination of medium- range ensemble precipitation forecasts , 2012 .

[84]  J. Bröcker Evaluating raw ensembles with the continuous ranked probability score , 2012 .

[85]  J. Ruiz,et al.  How sensitive are probabilistic precipitation forecasts to the choice of calibration algorithms and the ensemble generation method? Part I: sensitivity to calibration methods , 2012 .

[86]  Matthew S. Johnson,et al.  Probabilistic wind gust forecasting using nonhomogeneous Gaussian regression , 2012 .

[87]  J. Segers Asymptotics of empirical copula processes under non-restrictive smoothness assumptions , 2010, 1012.2133.

[88]  Emmanuel Roulin,et al.  Postprocessing of Ensemble Precipitation Predictions with Extended Logistic Regression Based on Hindcasts , 2012 .

[89]  Thordis L. Thorarinsdottir,et al.  Multivariate probabilistic forecasting using ensemble Bayesian model averaging and copulas , 2012, 1202.3956.

[90]  J. M. Sloughter,et al.  Probabilistic Wind Speed Forecasting Using Ensembles and Bayesian Model Averaging , 2010 .

[91]  T. Kneib Beyond mean regression , 2013 .

[92]  Z. B. Bouallègue Calibrated Short-Range Ensemble Precipitation Forecasts Using Extended Logistic Regression with Interaction Terms , 2013 .

[93]  M. Scheuerer Probabilistic quantitative precipitation forecasting using Ensemble Model Output Statistics , 2013, 1302.0893.

[94]  Pierre Pinson,et al.  Wind Energy: Forecasting Challenges for Its Operational Management , 2013, 1312.6471.

[95]  J. M. Sloughter,et al.  Probabilistic Wind Speed Forecasting Using Ensembles and Bayesian Model Averaging , 2013 .

[96]  S. Dogra,et al.  Summary of recommendations for leg ulcers , 2014, Indian dermatology online journal.