Predicting the forecast skill for the European region with the use of machine learning models

Users of Numerical Weather Prediction (NWP) services demand reliable weather forecasts, from short-range to seasonal lead times. Weather forecasts, such as from the European Centre for Medium-Range Weather Forecasts (ECMWF), have constantly improved. The forecast skill horizon has increased by one day per decade. Therefore, the 6 day forecast in 2010 was as good as the 3 day forecast in 1980. The predictability of the atmosphere is physically limited, which we call intrinsic predictability limit. This is due to the chaotic nature of the atmosphere. Small-scale uncertainties in the initial state may grow rapidly in time. The practical predictability limit, the time-scale at which actual forecast models still provide reliable forecasts, is far from reaching the intrinsic predictability limit. Forecast models are imperfect as they can not resolve all physical processes and use parameterisations. The intrinsic and practical predictability depend on the large-scale atmospheric state which can be described by different climate modes. Preferred climate modes are the Madden-Julian oscillation (MJO), stratospheric polar vortex (SPV) and North Atlantic oscillation (NAO), and their relation to each other, also called teleconnections. Thus, the overarching hypothesis of this thesis is that knowledge about the state of these climate modes at forecast initialisation time can help to predict the practical predictability of the atmosphere. Next to the atmospheric state, the forecast system itself can provide information about the reliability of its forecast, using the dispersion of individual forecast members in an ensemble forecast, hereafter referred to as ensemble spread. Our aim is to use the climate modes and the ensemble spread, which are known a priori, to predict the forecast skill of the ECMWF forecast for the European region. Our method involves a set of three machine learning (ML) models with different network structures, a fully connected neural network (FCNN), long short-term memory network (LSTM) and convolutional neural network (CNN). Applying the three trained ML models on the same period, the extended winter (November– March) from 2013 to 2016, for lead times up to 15 days, we find that the overall performance for all ML models is equally good. In comparison to two non-machine learning reference models, based on the climatology and the ensemble spread, our ML models are performing as good or better than the reference models. Especially at lead times of 4 to 11 days, the ML models are performing significantly better than the climatology. Using only the most confident predictions of our ML models significantly increases their skill. This effect is not visible for the climatolog-