Evaluating the performance of two global ensemble forecasting systems in predicting rainfall over India during the southwest monsoons

Ensemble prediction systems help in quantifying the inherent uncertainties in numerical weather prediction models. Verification of forecasts is essential, in order to monitor and improve forecast quality. Also, verification can be used to compare the capabilities of different numerical weather prediction models in predicting the weather. This study deals with a comparison of probabilistic rainfall forecasts obtained from the National Centre for Medium Range Weather Forecasting (NCMRWF) Global Ensemble Forecast System (NGEFS) and the UK Met Office Global and Regional Ensemble Prediction System (MOGREPS) for four monsoon seasons, June–September 2012–2015. Verification is done based on the Brier score, the Brier skill score, the reliability diagram, the relative operating characteristic (ROC) curve and the area under the ROC curve (AROC). The NMSG (India Meteorological Department NCMRWF merged satellite and gauge) observation dataset is used for verification. The Brier score values for verification of the MOGREPS are lower by approximately 6–14% across all rainfall thresholds and lead times, indicating that these forecasts match better with observations than the NGEFS. This is further reiterated by Brier skill score values of MOGREPS forecasts which are approximately 13–47% higher than the NGEFS. Furthermore, the reliability diagram shows that forecast probabilities are closer to observed frequencies for the MOGREPS than the NGEFS. The AROC for the MOGREPS is also higher than for the NGEFS, hence indicating a better skill of the MOGREPS. From this study, it can be concluded that the MOGREPS showed a better capability in predicting rainfall during the southwest monsoons of 2012–2015 over the Indian region compared to the NGEFS.

[1]  Nicholas E. Graham,et al.  Conditional Probabilities, Relative Operating Characteristics, and Relative Operating Levels , 1999 .

[2]  J. Swets.  The Relative Operating Characteristic in Psychology , 1973, Science.

[3]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[4]  Andrew C. Lorenc,et al.  Why does 4D‐Var beat 3D‐Var? , 2005 .

[5]  A. H. Murphy.  A New Vector Partition of the Probability Score , 1973 .

[6]  B. Parthasarathy,et al.  Fluctuations in All-India summer monsoon rainfall during 1871–1978 , 1984 .

[7]  Yuejian Zhu,et al.  Initial perturbations based on the ensemble transform (ET) technique in the NCEP global operational forecast system , 2008 .

[8]  R. Buizza,et al.  A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems , 2005 .

[9]  E. Lorenz.  The predictability of a flow which possesses many scales of motion , 1969 .

[10]  ASHIS K MITRA,et al.  Gridded daily Indian monsoon rainfall for 14 seasons: Merged TRMM and IMD gauge analyzed values , 2013, Journal of Earth System Science.

[11]  K. Sharma,et al.  Forecasting the heavy rainfall during Himalayan flooding—June 2013 , 2014 .

[12]  Yuejian Zhu,et al.  Evaluation of TIGGE ensemble predictions of Northern Hemisphere summer precipitation during 2008–2012 , 2014 .

[13]  A. Bohra,et al.  Ensemble spread and systematic errors in the medium-range predictions during the Indian summer monsoon , 2011 .

[14]  N. Graham,et al.  Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation , 2002 .

[15]  Craig H. Bishop,et al.  Adaptive sampling with the ensemble transform Kalman filter , 2001 .

[16]  D. S. Pai,et al.  Comparing two high‐resolution gauge‐adjusted multisatellite rainfall products over India for the southwest monsoon period , 2015 .

[17]  T. N. Krishnamurti,et al.  Daily Indian Precipitation Analysis Formed from a Merge of Rain-Gauge Data with the TRMM TMPA Satellite-Derived Rainfall Estimates , 2009 .

[18]  Daily rainfall analysis for Indian summer monsoon region , 1997 .