Remember the past: A comparison of time-adaptive training schemes for non-homogeneous regression

Abstract. Non-homogeneous regression is a frequently-used post-processing method for increasing the predictive skill of probabilistic ensemble weather forecasts. To adjust for seasonally varying error characteristics between ensemble forecasts and corresponding observations, different time-adaptive training schemes, including the classical sliding training window, have been developed for non-homogeneous regression. This study compares three such training approaches with the sliding-window approach for the application of post-processing near-surface air temperature forecasts across Central Europe. The predictive performance is evaluated conditional on three different groups of stations located in plains, in mountain foreland, and within mountainous terrain, as well as on changes in the ensemble forecast system of the European Centre for Medium-Range Weather Forecasts (ECMWF) used as input for the post-processing. The results show that time-adaptive training schemes using data over multiple years stabilize the temporal evolution of the coefficient estimates, yielding an increased predictive performance for all station types tested compared to the classical sliding-window approach based on the most recent days only. While this may not be surprising under fully stable model conditions, it is shown that remembering the past from multiple years of training data is typically also superior to the classical sliding-window when the ensemble prediction system is affected by certain model changes. Thus, reducing the variance of the non-homogeneous regression estimates due to increased training data appears to be more important than reducing its bias by adapting rapidly to the most current training data only.

[1]  Constantin Junk,et al.  Analog-Based Ensemble Model Output Statistics , 2015 .

[2]  Achim Zeileis,et al.  Heteroscedastic Censored and Truncated Regression with crch , 2016, R J..

[3]  Renate Hagedorn,et al.  Probabilistic Forecast Calibration Using ECMWF and GFS Ensemble Reforecasts. Part II: Precipitation , 2008 .

[4]  Achim Zeileis,et al.  Bivariate Gaussian models for wind vectors in a distributional regression framework , 2019, Advances in Statistical Climatology, Meteorology and Oceanography.

[5]  Annette Möller,et al.  Bivariate ensemble model output statistics approach for joint forecasting of wind speed and temperature , 2015, Meteorology and Atmospheric Physics.

[6]  Richard E. Chandler,et al.  New approaches to postprocessing of multi‐model ensemble forecasts , 2019, Quarterly Journal of the Royal Meteorological Society.

[7]  Jakob W. Messner,et al.  Estimation Methods for Nonhomogeneous Regression Models: Minimum Continuous Ranked Probability Score versus Maximum Likelihood , 2018, Monthly Weather Review.

[8]  Alan Y. Chiang,et al.  Generalized Additive Models: An Introduction With R , 2007, Technometrics.

[9]  S. Baran,et al.  Similarity‐based semilocal estimation of post‐processing models , 2015, 1509.03521.

[10]  Tilmann Gneiting,et al.  Skill of Global Raw and Postprocessed Ensemble Predictions of Rainfall over Northern Tropical Africa , 2017, 1708.04420.

[11]  Mark J. Rodwell,et al.  Flow-Dependent Reliability: A Path to More Skillful Ensemble Forecasts , 2018 .

[12]  A. Raftery,et al.  Calibrated Surface Temperature Forecasts from the Canadian Ensemble Prediction System Using Bayesian Model Averaging , 2007 .

[13]  Stephan Rasp,et al.  Neural networks for post-processing ensemble weather forecasts , 2018, Monthly Weather Review.

[14]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[15]  Claudia Czado,et al.  Vine copula based post-processing of ensemble forecasts for temperature , 2018, 1811.02255.

[16]  Olivier Mestre,et al.  Calibrated Ensemble Forecasts Using Quantile Regression Forests and Ensemble Model Output Statistics , 2016 .

[17]  M. Scheuerer Probabilistic quantitative precipitation forecasting using Ensemble Model Output Statistics , 2013, 1302.0893.

[18]  Alexander Jordan,et al.  Evaluating Probabilistic Forecasts with scoringRules , 2017, Journal of Statistical Software.

[19]  T. Hothorn,et al.  Distributional regression forests for probabilistic precipitation forecasting in complex terrain , 2018, The Annals of Applied Statistics.

[20]  Tim N. Palmer,et al.  The economic value of ensemble forecasts as a tool for risk assessment: From days to decades , 2002 .

[21]  S'andor Baran,et al.  Censored and shifted gamma distribution based EMOS model for probabilistic quantitative precipitation forecasting , 2015, 1512.04068.

[22]  Ulrich Corsmeier,et al.  Forecasting wind gusts in winter storms using a calibrated convection‐permitting ensemble , 2018, Quarterly Journal of the Royal Meteorological Society.

[23]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[24]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[25]  Tilmann Gneiting,et al.  Isotonic distributional regression , 2021, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[26]  Achim Zeileis,et al.  BAMLSS: Bayesian Additive Models for Location, Scale, and Shape (and Beyond) , 2018, Journal of Computational and Graphical Statistics.

[27]  Achim Zeileis,et al.  Ensemble Post-Processing of Daily Precipitation Sums over Complex Terrain Using Censored High-Resolution Standardized Anomalies , 2017 .

[28]  Florian Pappenberger,et al.  Discrete Postprocessing of Total Cloud Cover Ensemble Forecasts , 2016 .

[29]  Achim Zeileis,et al.  Spatio‐temporal precipitation climatology over complex terrain using a censored additive regression model , 2016, International journal of climatology : a journal of the Royal Meteorological Society.

[30]  Nadja Klein,et al.  Bayesian structured additive distributional regression for multivariate responses , 2015 .

[31]  Jonathan Demaeyer,et al.  Correcting for model changes in statistical postprocessing – an approach based on response theory , 2019, Nonlinear Processes in Geophysics.