A posteriori clear-sky identification methods in solar irradiance time series: Review and preliminary validation using sky imagers

Abstract This study examines all known methods that have been proposed in the literature to identify clear-sky periods in historical solar irradiance time series. Two different types of clear-sky detection (CSD) methods are discussed: those (16 total) that attempt to isolate periods of 1-min or more cloudless conditions, and those (5 total) that only attempt to detect clear-sun periods. All methods are found to rely on a diversity of inputs and on a variety of tests that typically examine the smoothness of the temporal variation of global and/or direct irradiance. Using samples of a few days with variable cloudiness, it is shown that these methods all have obvious strengths and weaknesses. Although this justifies a detailed validation to determine which method(s) could be best suited in the practice of solar radiation modeling or other applications, the current lack of appropriate equipment at high-quality reference radiometric stations prevents such an endeavor. Only a preliminary study is conducted here at seven stations of the SURFRAD network in the U.S., where 1-min irradiance measurements are available, along with sky data from a Total Sky Imager (TSI). The many limitations of the latter prevent its data to be considered “ground truth” here. Nevertheless, the comparison of the results from all CSD methods and 1.2 million TSI observations from all SURFRAD sites provides important qualitative and quantitative information, using a variety of performance indicators. Overall, two CSD methods appear more robust and are recommended, pending better high-resolution and high-performance cloud observations from modern sky cameras to redo these tests.

[1]  S. S. Jagtap,et al.  Analysis of solar radiation data for Port Harcourt, Nigeria , 1992 .

[2]  E. Kassianov,et al.  Cloud Area Distributions of Shallow Cumuli: A New Method for Ground-Based Images , 2018, Atmosphere.

[3]  Rolf Philipona,et al.  The clear‐sky index to separate clear‐sky from cloudy‐sky situations in climate research , 2000 .

[4]  A. A. Silva Local cloud cover, ground-based and satellite measurements of erythemal dose rate for an urban, tropical site in Southern Hemisphere , 2011 .

[5]  Christian A. Gueymard,et al.  AEROSOL TURBIDITY DERIVATION FROM BROADBAND IRRADIANCE MEASUREMENTS: METHODOLOGICAL ADVANCES AND UNCERTAINTY ANALYSIS , 2013 .

[6]  L. Alados-Arboledas,et al.  Cloud screening and quality control algorithm for star photometer data: assessment with lidar measurements and with all-sky images , 2012 .

[7]  L. Wald,et al.  Can AERONET data be used to accurately model the monochromatic beam and circumsolar irradiances under cloud-free conditions in desert environment? , 2015 .

[8]  Stéphane Grieu,et al.  Cloud Detection Methodology Based on a Sky-imaging System☆ , 2015 .

[9]  C. Gueymard,et al.  The national solar radiation data base (NSRDB) for CSP applications , 2019, SOLARPACES 2018: International Conference on Concentrating Solar Power and Chemical Energy Systems.

[10]  Kanjian Zhang,et al.  A Data-driven Clear Sky Model for Direct Normal Irradiance , 2018, Journal of Physics: Conference Series.

[11]  J. M. Vindel,et al.  Correcting satellite derived DNI with systematic and seasonal deviations: Application to India , 2015 .

[12]  M. Haeffelin,et al.  ELIFAN, an algorithm for the estimation of cloud cover from sky imagers , 2019 .

[13]  B. Hodge,et al.  A stochastic downscaling approach for generating high-frequency solar irradiance scenarios , 2018, Solar Energy.

[14]  C. Gueymard,et al.  Evaluation of conventional and high-performance routine solar radiation measurements for improved solar resource, climatological trends, and radiative modeling , 2009 .

[15]  P. Ineichen,et al.  A new simplified version of the perez diffuse irradiance model for tilted surfaces , 1987 .

[16]  Julien Eynard,et al.  A new approach to the real-time assessment and intraday forecasting of clear-sky direct normal irradiance , 2018, Solar Energy.

[17]  Christopher J. Smith,et al.  Cloud cover effect of clear-sky index distributions and differences between human and automatic cloud observations , 2017 .

[18]  Rolf Philipona,et al.  Automatic cloud amount detection by surface longwave downward radiation measurements , 2004 .

[19]  Dong Huang,et al.  3D cloud detection and tracking system for solar forecast using multiple sky imagers , 2015 .

[20]  Estimating clear-day solar radiation: An evaluation of three models , 1983 .

[21]  C. Gueymard Clear-sky irradiance predictions for solar resource mapping and large-scale applications: Improved validation methodology and detailed performance analysis of 18 broadband radiative models , 2012 .

[22]  Jin Xu,et al.  Cloud motion estimation for short term solar irradiation prediction , 2013, 2013 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[23]  Jan Kleissl,et al.  A synthetic, spatially decorrelating solar irradiance generator and application to a LV grid model with high PV penetration , 2017 .

[24]  C. Gueymard,et al.  Long-Term Variability of Aerosol Optical Depth, Dust Episodes, and Direct Normal Irradiance Over Kuwait for CSP Applications , 2017 .

[25]  Johannes W. Kaiser,et al.  McClear: a new model estimating downwelling solar radiation at ground level in clear-sky conditions , 2013 .

[26]  Aron Habte,et al.  Evaluation of the National Solar Radiation Database (NSRDB): 1998-2015 , 2017 .

[27]  J. Kleissl,et al.  Intra-hour forecasting with a total sky imager at the UC San Diego solar energy testbed , 2011 .

[28]  E. Clothiaux,et al.  Uncertainties in modeled and measured clear‐sky surface shortwave irradiances , 1997 .

[29]  C. Gueymard REST2: High-performance solar radiation model for cloudless-sky irradiance, illuminance, and photosynthetically active radiation – Validation with a benchmark dataset , 2008 .

[30]  L. Alados-Arboledas,et al.  Comparison of Cloudless Sky Parameterizations of Solar Irradianceat Various Spanish Midlatitude Locations , 2000 .

[31]  C. Gueymard Cloud and albedo enhancement impacts on solar irradiance using high-frequency measurements from thermopile and photodiode radiometers. Part 1: Impacts on global horizontal irradiance , 2017 .

[32]  Dazhi Yang,et al.  A correct validation of the National Solar Radiation Data Base (NSRDB) , 2018, Renewable and Sustainable Energy Reviews.

[33]  José A. Ruiz-Arias,et al.  Worldwide inter-comparison of clear-sky solar radiation models: Consensus-based review of direct and global irradiance components simulated at the earth surface , 2018, Solar Energy.

[34]  Daniel Rowe,et al.  Short-term irradiance forecasting using skycams: Motivation and development , 2014 .

[35]  A. Henderson‐sellers,et al.  Almost a Century of “Imaging” Clouds Over the Whole-Sky Dome , 1989 .

[36]  C. Long,et al.  Total Sky Imager Model 880 Status and Testing Results , 2001 .

[37]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[38]  Xinlei Han,et al.  First assessment of surface solar irradiance derived from Himawari-8 across China , 2018, Solar Energy.

[39]  J. A. Ruiz-Arias,et al.  An advanced ANN-based method to estimate hourly solar radiation from multi-spectral MSG imagery , 2015 .

[40]  Anubhav Jain,et al.  Automatic Detection of Clear-sky Periods Using Ground and Satellite Based Solar Resource Data , 2018, 2018 IEEE 7th World Conference on Photovoltaic Energy Conversion (WCPEC) (A Joint Conference of 45th IEEE PVSC, 28th PVSEC & 34th EU PVSEC).

[41]  D. W. Nelson,et al.  Optimal Measurement of Surface Shortwave Irradiance Using Current Instrumentation , 1997 .

[42]  J. Michalsky,et al.  Modeling daylight availability and irradiance components from direct and global irradiance , 1990 .

[43]  R. McKenzie,et al.  Two Methods for Retrieving UV Index for All Cloud Conditions from Sky Imager Products or Total SW Radiation Measurements , 2014, Photochemistry and photobiology.

[44]  Nicholas A. Engerer,et al.  A tuning routine to correct systematic influences in reference PV systems' power outputs , 2017 .

[45]  L. Ramírez,et al.  Angstrom turbidity and ozone column estimations from spectral solar irradiance in a semi-desertic environment in Spain , 2009 .

[46]  C. Gueymard,et al.  Reducing Uncertainties in Large-Scale Solar Resource Data: The Impact of Aerosols , 2018, IEEE Journal of Photovoltaics.

[47]  K. Gaustad,et al.  The Shortwave (SW) Clear-Sky Detection and Fitting Algorithm: Algorithm Operational Details and Explanations , 2004 .

[48]  Clifford W. Hansen,et al.  Identification of periods of clear sky irradiance in time series of GHI measurements , 2016 .

[49]  Carlos F.M. Coimbra,et al.  Forecasting of Global Horizontal Irradiance Using Sky Cover Indices , 2013 .

[50]  Yu Xie,et al.  A new approach for simultaneously retrieving cloud albedo and cloud fraction from surface-based shortwave radiation measurements , 2013 .

[51]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[52]  Victoria E. Cachorro,et al.  Solar radiation measurements compared to simulations at the BSRN Izaña station. Mineral dust radiative forcing and efficiency study , 2014 .

[53]  J. Kleissl,et al.  Development of a sky imaging system for short-term solar power forecasting , 2014 .

[54]  A. Louche,et al.  Determination of Ångström's turbidity coefficient from direct total solar irradiance measurements , 1987 .

[55]  P. Ineichen Comparison of eight clear sky broadband models against 16 independent data banks , 2006 .

[56]  Gustau Camps-Valls,et al.  Multitemporal Cloud Masking in the Google Earth Engine , 2018, Remote. Sens..

[57]  Charles N. Long,et al.  An Automated Quality Assessment and Control Algorithm for Surface Radiation Measurements , 2008 .

[58]  Nicholas A. Engerer,et al.  Validating nine clear sky radiation models in Australia , 2015 .

[59]  Nicholas A. Engerer,et al.  KPV: A clear-sky index for photovoltaics , 2014 .

[60]  Bernhard Geiger,et al.  Satellite Application Facilities irradiance products: hourly time step comparison and validation over Europe , 2009 .

[61]  Michael J. Foster,et al.  A comparison of multiscale variations of decade‐long cloud fractions from six different platforms over the Southern Great Plains in the United States , 2014 .

[62]  B. Matthews Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[63]  Krista Gaustad,et al.  Estimation of fractional sky cover from broadband shortwave radiometer measurements , 2006 .

[64]  John S. Uebersax,et al.  Diversity of decision-making models and the measurement of interrater agreement. , 1987 .

[65]  P. Ineichen Validation of models that estimate the clear sky global and beam solar irradiance , 2016 .

[66]  Charles N. Long,et al.  Correcting for Circumsolar and Near-Horizon Errors in Sky Cover Retrievals from Sky Images , 2010 .

[67]  Charles N. Long,et al.  Retrievals of cloud fraction and cloud albedo from surface‐based shortwave radiation measurements: A comparison of 16 year measurements , 2014 .

[68]  C. Coimbra,et al.  Intra-hour DNI forecasting based on cloud tracking image analysis , 2013 .

[69]  J. A. Ruiz-Arias,et al.  A multi-model benchmarking of direct and global clear-sky solar irradiance predictions at arid sites using a reference physical radiative transfer model , 2018, Solar Energy.

[70]  J. Garrison,et al.  Critical evaluation of precipitable water and atmospheric turbidity in Canada using measured hourly solar irradiance , 1998 .

[71]  Clifford W. Hansen,et al.  Global horizontal irradiance clear sky models : implementation and analysis. , 2012 .

[72]  R. Inman,et al.  Impact of local broadband turbidity estimation on forecasting of clear sky direct normal irradiance , 2015 .

[73]  C. Long,et al.  Identification of clear skies from broadband pyranometer measurements and calculation of downwelling shortwave cloud effects , 2000 .

[74]  Christian A. Gueymard,et al.  Turbidity Determination from Broadband Irradiance Measurements: A Detailed Multicoefficient Approach , 1998 .

[75]  M. Alia-Martinez,et al.  Benchmark of algorithms for solar clear-sky detection , 2016 .

[76]  Viorel Badescu,et al.  Accuracy and sensitivity analysis for 54 models of computing hourly diffuse solar irradiation on clear sky , 2013, Theoretical and Applied Climatology.

[77]  J. A. Ruiz-Arias,et al.  Validation of direct normal irradiance predictions under arid conditions: A review of radiative models and their turbidity-dependent performance , 2015 .

[78]  Kaicun Wang,et al.  A Comparison of MODIS-Derived Cloud Fraction with Surface Observations at Five SURFRAD Sites , 2015 .

[79]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[80]  J. Shields,et al.  The Whole Sky Imager - A Year of Progress , 1998 .

[81]  L. Zarzalejo,et al.  Estimation of daily Linke turbidity factor by using global irradiance measurements at solar noon , 2009 .

[82]  Chao Li,et al.  Detection of cloud cover using dynamic thresholds and radiative transfer models from the polarization satellite image , 2019, Journal of Quantitative Spectroscopy and Radiative Transfer.

[83]  Nicholas A. Engerer,et al.  QCPV: A quality control algorithm for distributed photovoltaic array power output , 2017 .

[84]  J. M. Vindel,et al.  Sensitivity of satellite-based methods for deriving solar radiation to different choice of aerosol input and models , 2014 .

[85]  Christian A. Gueymard,et al.  Temporal variability in direct and global irradiance at various time scales as affected by aerosols , 2012 .

[86]  Viorel Badescu,et al.  Computing global and diffuse solar hourly irradiation on clear sky. Review and testing of 54 models , 2012 .

[87]  J. Alonso,et al.  Determination of the sun area in sky camera images using radiometric data , 2014 .

[88]  Taiping Zhang,et al.  Assessment of BSRN radiation records for the computation of monthly means , 2010 .

[89]  Kanjian Zhang,et al.  Real-time Clear-sky Model and Cloud Cover for Direct Normal Irradiance Prediction , 2018, Journal of Physics: Conference Series.

[90]  A. Louche,et al.  Experimental verification of some clear-sky insolation models , 1988 .

[91]  D. Djafer,et al.  Identification of clear days from solar irradiance observations using a new method based on the wavelet transform , 2017 .

[92]  Josep Calbó,et al.  Retrieving Cloud Characteristics from Ground-Based Daytime Color All-Sky Images , 2006 .

[93]  Robert Pitz-Paal,et al.  Direct normal irradiance related definitions and applications: The circumsolar issue , 2014 .

[94]  Janet E. Shields,et al.  Day/night whole sky imagers for 24-h cloud and sky assessment: history and overview. , 2013, Applied optics.

[95]  Xin Li,et al.  An integrated approach to estimate shortwave solar radiation on clear-sky days in rugged terrain using MODIS atmospheric products , 2015 .

[96]  Jan Kleissl,et al.  Comparison of Solar Power Output Forecasting Performance of the Total Sky Imager and the University of California, San Diego Sky Imager , 2014 .

[97]  Frank Vignola,et al.  Determination of atmospheric turbidity from the diffuse-beam broadband irradiance ratio , 1998 .

[98]  Matthew J. Reno,et al.  Identifying periods of clear sky direct normal irradiance , 2017 .

[99]  J. A. Ruiz-Arias,et al.  Direct normal irradiance modeling: Evaluating the impact on accuracy of worldwide gridded aerosol databases , 2019, SOLARPACES 2018: International Conference on Concentrating Solar Power and Chemical Energy Systems.

[100]  C. W. Chow,et al.  A method for cloud detection and opacity classification based on ground based sky imagery , 2012 .

[101]  C. Long,et al.  Cloud Coverage Based on All-Sky Imaging and Its Impact on Surface Solar Irradiance , 2003 .

[102]  Lucas Alados-Arboledas,et al.  Multi-exposure adaptive threshold technique for cloud detection with sky imagers , 2015 .