Verification of solar irradiance probabilistic forecasts

Abstract We propose a framework for evaluating the quality of solar irradiance probabilistic forecasts. The verification framework is based on visual diagnostic tools and a set of scoring rules mostly originating from the weather forecast verification community. Two types of probabilistic forecasts are used as a basis to illustrate the application of these verification approaches. The first one consists in ensemble forecasts commonly provided by national or international meteorological centres. The second one originates from statistical methods and produces a set of discrete quantile forecasts, the nominal proportions of which span the unit interval. These probabilistic forecasts are evaluated for two selected sites that experience very different climatic conditions. The first site is located in the continental US while the second one is situated on La Reunion Island. Although visual diagnostic tools can help identify deficiencies in generated forecasts, it is recommended that a set of numerical scores be used to assess the quality of probabilistic forecasts. In particular, the Continuous Ranked Probability Score (CRPS) seems to have all the features needed to evaluate a probabilistic forecasting system and, as such, may become a standard for verifying solar irradiance probabilistic forecasts and by extension probabilistic forecasts of solar power generation.

[1]  John Boland,et al.  Nonparametric short-term probabilistic forecasting for solar radiation , 2016 .

[2]  Dazhi Yang,et al.  A universal benchmarking method for probabilistic solar irradiance forecasting , 2019, Solar Energy.

[3]  J. Kleissl,et al.  Chapter 8 – Overview of Solar-Forecasting Methods and a Metric for Accuracy Evaluation , 2013 .

[4]  John Boland,et al.  Forecasting solar radiation on an hourly time scale using a Coupled AutoRegressive and Dynamical System (CARDS) model , 2013 .

[5]  R. Koenker,et al.  Regression Quantiles , 2007 .

[6]  Henrik Madsen,et al.  Short-term probabilistic forecasting of wind speed using stochastic differential equations , 2016 .

[7]  Hadrien Verbois,et al.  Probabilistic forecasting of day-ahead solar irradiance using quantile gradient boosting , 2018, Solar Energy.

[8]  mith,et al.  Evaluating Probabilistic Forecasts Using Information Theory , 2002 .

[9]  Nick van de Giesen,et al.  Kullback–Leibler Divergence as a Forecast Skill Score with Classic Reliability–Resolution–Uncertainty Decomposition , 2010 .

[10]  I. Jolliffe,et al.  Forecast verification : a practitioner's guide in atmospheric science , 2011 .

[11]  Mathieu David,et al.  Probabilistic Solar Forecasting Using Quantile Regression Models , 2017 .

[12]  Leonard A. Smith,et al.  Scoring Probabilistic Forecasts: The Importance of Being Proper , 2007 .

[13]  Gilles Notton,et al.  Forecasting method for global radiation time series without training phase: Comparison with other well-known prediction methodologies , 2017 .

[14]  Tao Zhang,et al.  Interval prediction of solar power using an Improved Bootstrap method , 2018 .

[15]  Petra Friederichs,et al.  Decomposition and graphical portrayal of the quantile score , 2014 .

[16]  Robert P. Broadwater,et al.  Current status and future advances for wind speed and power forecasting , 2014 .

[17]  R. L. Winkler.  A Decision-Theoretic Approach to Interval Estimation , 1972 .

[18]  Tim N. Palmer,et al.  Ensemble forecasting , 2008, J. Comput. Phys..

[19]  D. Boldo,et al.  Very short term forecasting of the Global Horizontal Irradiance using a spatio-temporal autoregressive model , 2014 .

[20]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[21]  J. Kleissl,et al.  Reporting of irradiance modeling relative prediction errors , 2013 .

[22]  Leonard A. Smith,et al.  Evaluating Probabilistic Forecasts Using Information Theory , 2002 .

[23]  Detlev Heinemann,et al.  Prediction of Solar Irradiance and Photovoltaic Power , 2012 .

[24]  S. Nahavandi,et al.  Prediction Intervals for Short-Term Wind Farm Power Generation Forecasts , 2013, IEEE Transactions on Sustainable Energy.

[25]  P. Pinson,et al.  Generation and evaluation of space–time trajectories of photovoltaic power , 2016, 1603.06649.

[26]  T. Soubdhan,et al.  A benchmarking of machine learning techniques for solar radiation forecasting in an insular context , 2015 .

[27]  T. Hamill.  Interpretation of Rank Histograms for Verifying Ensemble Forecasts , 2001 .

[28]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[29]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[30]  Pierre Pinson,et al.  Non‐parametric probabilistic forecasts of wind power: required properties and evaluation , 2007 .

[31]  Zied Ben Bouallegue.  Assessment and added value estimation of an ensemble approach with a focus on global radiation forecasts , 2015 .

[32]  Pierre Pinson,et al.  Very Short-Term Nonparametric Probabilistic Forecasting of Renewable Energy Generation— With Application to Solar Energy , 2016, IEEE Transactions on Power Systems.

[33]  J. Bröcker.  Evaluating raw ensembles with the continuous ranked probability score , 2012 .

[34]  Jeffrey L. Anderson.  A Method for Producing and Evaluating Probabilistic Forecasts from Ensemble Model Integrations , 1996 .

[35]  Philippe Lauret,et al.  Probabilistic forecasting of the solar irradiance with recursive ARMA and GARCH models , 2016 .

[36]  H. Madsen,et al.  Reliability diagrams for non‐parametric density forecasts of continuous variables: Accounting for serial correlation , 2010 .

[37]  Gordon Reikard.  Predicting solar radiation at high resolutions: A comparison of time series forecasts , 2009 .

[38]  Mario Paolone,et al.  Irradiance prediction intervals for PV stochastic generation in microgrid applications , 2016 .

[39]  L. D. Monache,et al.  An analog ensemble for short-term probabilistic solar power forecast , 2015 .

[40]  Olivier Pannekoucke,et al.  A benchmark of statistical regression methods for short-term forecasting of photovoltaic electricity production, part I: Deterministic forecast of hourly production , 2014 .

[41]  Bodo Ahrens,et al.  Generalization of the Ignorance Score: Continuous Ranked Version and Its Decomposition , 2012 .

[42]  Carlos F.M. Coimbra,et al.  Assessment of machine learning techniques for deterministic and probabilistic intra-hour solar forecasts , 2018, Renewable Energy.

[43]  L. D. Monache,et al.  An application of the ECMWF Ensemble Prediction System for short-term solar power forecasting , 2016 .

[44]  Rob J Hyndman,et al.  Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond , 2016 .

[45]  Carlos F.M. Coimbra,et al.  Real-time prediction intervals for intra-hour DNI forecasts , 2015 .

[46]  H. Hersbach.  Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems , 2000 .

[47]  Carlos F.M. Coimbra,et al.  History and trends in solar irradiance and PV power forecasting: A preliminary assessment and review using text mining , 2018, Solar Energy.

[48]  Carlos F.M. Coimbra,et al.  Nearest-neighbor methodology for prediction of intra-hour global horizontal and direct normal irradiances , 2015 .

[49]  Richard Perez,et al.  Modeling PV fleet output variability , 2012 .

[50]  Leonard A. Smith,et al.  Increasing the Reliability of Reliability Diagrams , 2007 .

[51]  Carlos F.M. Coimbra,et al.  Short-term probabilistic forecasts for Direct Normal Irradiance , 2017 .

[52]  Joakim Widén,et al.  Review on probabilistic forecasting of photovoltaic power production and electricity consumption , 2018 .

[53]  Pierre Pinson,et al.  Probabilistic forecasting of the wave energy flux , 2012, Applied Energy.

[54]  Mathieu David,et al.  Comparison of intraday probabilistic forecasting of solar irradiance using only endogenous data , 2018, International Journal of Forecasting.

[55]  A. H. Murphy,et al.  What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting , 1993 .

[56]  Daniel S. Wilks,et al.  Statistical Methods in the Atmospheric Sciences: An Introduction , 1995 .

[57]  C. Coimbra,et al.  Forecasting of global and direct solar irradiance using stochastic learning methods, ground experiments and the NWS database , 2011 .

[58]  Pierre Pinson,et al.  Discussion of “Prediction Intervals for Short-Term Wind Farm Generation Forecasts” and “Combined Nonparametric Prediction Intervals for Wind Power Generation” , 2014 .