Reliability of Regression-Corrected Climate Forecasts

AbstractRegression is often used to calibrate climate model forecasts with observations. Reliability is an aspect of forecast quality that refers to the degree of correspondence between forecast probabilities and observed frequencies of occurrence. While regression-corrected climate forecasts are reliable in principle, the estimated regression parameters used in practice are affected by sampling error. The low skill and small sample sizes typically encountered in climate prediction imply substantial sampling error in the estimated regression parameters. Here the reliability of regression-corrected climate forecasts is analyzed for the case of joint-Gaussian distributed ensemble forecasts and observations with regression parameters estimated by least squares. Hypothesis testing of the regression parameters provides direct information about the skill and reliability of the uncorrected ensemble-based probability forecasts. However, the regression-corrected probability forecasts with estimated parameters are ...

[1]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[2]  Michael K. Tippett,et al.  Estimation of Seasonal Precipitation Tercile-Based Categorical Probabilities from Ensembles , 2007 .

[3]  A. Barnston,et al.  A Degeneracy in Cross-Validated Skill in Regression-based Forecasts , 1993 .

[4]  T. DelSole,et al.  Field Significance of Regression Patterns , 2011 .

[5]  P. Xie,et al.  A Gauge-Based Analysis of Daily Precipitation over East Asia , 2007 .

[6]  Cécile Penland,et al.  Prediction of Nino 3 sea surface temperatures using linear inverse modeling , 1993 .

[7]  Neill E. Bowler,et al.  On the Reliability and Calibration of Ensemble Forecasts , 2009 .

[8]  Mark A. Liniger,et al.  The discrete brier and ranked probability skill scores , 2007 .

[9]  A. H. Murphy,et al.  Reliability of Subjective Probability Forecasts of Precipitation and Temperature , 1977 .

[10]  A. H. Murphy.  A New Vector Partition of the Probability Score , 1973 .

[11]  Scale-Selective Ridge Regression for Multimodel Forecasting , 2013 .

[12]  M. R. Allen,et al.  Checking for model consistency in optimal fingerprinting , 1999 .

[13]  Nicholas E. Graham,et al.  Conditional Exceedance Probabilities , 2007 .

[14]  A. Barnston,et al.  The North American multimodel ensemble: Phase-1 seasonal-to-interannual prediction; phase-2 toward developing intraseasonal prediction , 2014 .

[15]  H. Glahn,et al.  The Use of Model Output Statistics (MOS) in Objective Weather Forecasting , 1972 .

[16]  Timothy DelSole,et al.  A Bayesian Framework for Multimodel Regression , 2007 .

[17]  A. Barnston,et al.  Statistical–Dynamical Seasonal Forecasts of Central-Southwest Asian Winter Precipitation , 2005 .

[18]  T. Hamill.  Interpretation of Rank Histograms for Verifying Ensemble Forecasts , 2001 .

[19]  P. Stott,et al.  Estimating signal amplitudes in optimal fingerprinting, part I: theory , 2003 .

[20]  Francis W. Zwiers,et al.  Improved Seasonal Probability Forecasts , 2003 .

[21]  J. Copas.  Regression, Prediction and Shrinkage , 1983 .

[22]  R. E. Livezey,et al.  Statistical Field Significance and its Determination by Monte Carlo Techniques , 1983 .

[23]  Timothy DelSole,et al.  Predictability and Information Theory. Part II: Imperfect Forecasts , 2005 .

[24]  Reason L. Machete,et al.  Early Warning with Calibrated and Sharper Probabilistic Forecasts , 2011, 1112.6390.

[25]  Willem A. Landman,et al.  Statistical Recalibration of GCM Forecasts over Southern Africa Using Model Output Statistics , 2002 .

[26]  Hui Wang,et al.  Statistical–Dynamical Predictions of Seasonal North Atlantic Hurricane Activity , 2011 .

[27]  V. Kousky,et al.  Assessing objective techniques for gauge‐based analyses of global daily precipitation , 2008 .

[28]  David S. Richardson,et al.  Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size , 2001 .

[29]  M. Iredell,et al.  The NCEP Climate Forecast System Version 2 , 2014 .

[30]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.