Finite Samples and Uncertainty Estimates for Skill Measures for Seasonal Prediction

Abstract The expected value for various measures of skill for seasonal climate predictions is determined by the signal-to-noise ratio. The expected value, however, is only realized for long verification time series. In practice, the verifications for specific seasons—for example, forecasts for the December–February seasonal mean—seldom exceed a sample size of 30. The estimates of skill measure based on small verification time series, because of sampling errors, can have large departures from their expected value. An analysis of spread in the estimates of skill measures with the length of verification time series and for different signal-to-noise ratios is made. The analysis is based on the Monte Carlo approach and skill measures for deterministic, categorical, and probabilistic forecasts are considered. It is shown that the behavior of spread for various skill measures can be very different and it is not always the largest for the small values of signal-to-noise ratios.

[1]  R. Wu,et al.  Changes in Spread and Predictability Associated with ENSO in an Ensemble Coupled GCM , 2006 .

[2]  M. Tippett,et al.  A Predictability Measure Applied to Seasonal Predictions of the Arctic Oscillation , 2007 .

[3]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[4]  Arun Kumar,et al.  Seasonal Predictions, Probabilistic Verifications, and Ensemble Size , 2001 .

[5]  Arun Kumar.  On the Interpretation and Utility of Skill Information for Seasonal Climate Predictions , 2007 .

[6]  C. Ropelewski,et al.  Global and Regional Scale Precipitation Patterns Associated with the El Niño/Southern Oscillation , 1987 .

[7]  Prashant D. Sardeshmukh,et al.  Changes of Probability Associated with El Niño , 2000 .

[8]  M. Hoerling,et al.  Prospects and Limitations of Seasonal Atmospheric GCM Predictions , 1995 .

[9]  Zoltan Toth,et al.  Why Do Forecasts for “Near Normal” Often Fail? , 1991 .

[10]  Michael K. Tippett,et al.  Measuring the potential utility of seasonal climate predictions , 2004 .

[11]  M. Hoerling,et al.  Analysis of a Conceptual Model of Seasonal Climate Variability and Implications for Seasonal Prediction , 2000 .

[12]  Kevin E. Trenberth,et al.  Progress during TOGA in understanding and modeling global teleconnections associated with tropical sea surface temperatures , 1998 .

[13]  A. Barnston,et al.  Changes in the Spread of the Variability of the Seasonal Mean Atmospheric States Associated with ENSO , 2000 .

[14]  A. Barnston.  Correspondence among the correlation, RMSE, and Heidke forecast verification measures; refinement of the Heidke score , 1992 .