Ultra-fast preselection in lasso-type spatio-temporal solar forecasting problems

Abstract Solar forecasting using data collected by satellites or sensor networks is often framed as a many-predictor spatio-temporal regression problem. Whereas the regressand is the irradiance at the focal location, the regressors could be the irradiance at any neighboring location with any time lag. Lasso—the least absolute shrinkage and selection operator—is commonly used for regressor selection and regularization in order to enhance the forecast accuracy and interpretability of the regression model. However, when the number of regressors is much larger than the number of samples, lasso-type regressions are limited by the curse of dimensionality and the insufficient degree of freedom. The ultra-fast preselection algorithm herein proposed provides a remedy to the above-mentioned problem. The algorithm uses (probably) the world’s fastest similarity search routine, and preselects a user-defined number of most-relevant regressors based on the z-normalized Euclidean distance. The preselected regressors are then fed to lasso for forecasting. The algorithm is completely free from the curse of dimensionality, and does not require any meteorological prior, such as wind information or cloud vector field. It is customizable for other forecasting methods, such as vector autoregression or kriging, and can be implemented in just 10 lines of code.

[1]  W. V. Sark,et al.  Short-term peer-to-peer solar forecasting in a network of photovoltaic systems , 2017 .

[2]  Nan Chen,et al.  Solar irradiance forecasting using spatial-temporal covariance structures and time-forward kriging , 2013 .

[3]  Dazhi Yang,et al.  Reconciling solar forecasts: Geographical hierarchy , 2017 .

[4]  Peter Bühlmann.  Regression shrinkage and selection via the Lasso: a retrospective (Robert Tibshirani): Comments on the presentation , 2011 .

[5]  R. Inman,et al.  Solar forecasting methods for renewable energy integration , 2013 .

[6]  Dazhi Yang,et al.  Very short-term irradiance forecasting at unobserved locations using spatio-temporal kriging , 2015 .

[7]  Galen Maclaurin,et al.  The National Solar Radiation Data Base (NSRDB) , 2017, Renewable and Sustainable Energy Reviews.

[8]  Carlos D. Rodriguez-Gallegos,et al.  Reconciling solar forecasts: Temporal hierarchy , 2017 .

[9]  Laura M. Hinkelman,et al.  Differences between along-wind and cross-wind solar irradiance variability on small spatial scales , 2013 .

[10]  Michael Schweinberger,et al.  High-Dimensional Multivariate Time Series With Additional Structure , 2015, 1510.02159.

[11]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[12]  A. Cronin,et al.  Intra-hour forecasts of solar power production using measurements from a network of irradiance sensors , 2013 .

[13]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[14]  Sue Ellen Haupt,et al.  Evaluation of statistical learning configurations for gridded solar irradiance forecasting , 2017 .

[15]  R. Urraca,et al.  Review of photovoltaic power forecasting , 2016 .

[16]  Bin Yu,et al.  On Model Selection Consistency of the Elastic Net When p >> n , 2008 .

[17]  Joakim Widén,et al.  Review on probabilistic forecasting of photovoltaic power production and electricity consumption , 2018 .

[18]  Miguel Brito,et al.  Impact of network layout and time resolution on spatio-temporal solar forecasting , 2018 .

[19]  Jan Kleissl,et al.  A synthetic, spatially decorrelating solar irradiance generator and application to a LV grid model with high PV penetration , 2017 .

[20]  Jan Kleissl,et al.  Spatiotemporal interpolation and forecast of irradiance data using Kriging , 2017 .

[21]  Preface of Special issue: Progress in Solar Resource Assessment and Forecasting , 2018, Solar Energy.

[22]  Dazhi Yang,et al.  On adding and removing sensors in a solar irradiance monitoring network for areal forecasting and PV system performance evaluation , 2017 .

[23]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[24]  Grzegorz Dudek.  Pattern-based local linear regression models for short-term load forecasting , 2016 .

[25]  Ricardo J. Bessa,et al.  LASSO vector autoregression structures for very short-term wind power forecasting , 2017 .

[26]  Henrik Madsen,et al.  Multi-site solar power forecasting using gradient boosted regression trees , 2017 .

[27]  E. Arias-Castro,et al.  A Poisson model for anisotropic solar ramp rate correlations , 2014 .

[28]  Jan Kleissl,et al.  Editorial: Submission of Data Article is now open , 2018, Solar Energy.

[29]  Florian Ziel,et al.  Lasso estimation for GEFCom2014 probabilistic electric load forecasting , 2016, 1603.01376.

[30]  Stefan Feuerriegel,et al.  Putting Big Data analytics to work: Feature selection for forecasting electricity prices using the LASSO and random forests , 2014, J. Decis. Syst..

[31]  R. Tibshirani.  The Lasso Problem and Uniqueness , 2012, 1206.0313.

[32]  R. Tibshirani,et al.  Degrees of freedom in lasso problems , 2011, 1111.0653.

[33]  P. Blanc,et al.  Towards a standardized procedure to assess solar forecast accuracy: A new ramp and time alignment metric , 2017 .

[34]  Soteris A. Kalogirou,et al.  Machine learning methods for solar radiation forecasting: A review , 2017 .

[35]  Thomas Reindl,et al.  Solar irradiance forecasting using spatio-temporal empirical kriging and vector autoregressive models with parameter shrinkage , 2014 .

[36]  Dazhi Yang,et al.  Very short term irradiance forecasting using the lasso , 2015 .

[37]  Florian Ziel,et al.  Forecasting Electricity Spot Prices Using Lasso: On Capturing the Autoregressive Intraday Structure , 2015, IEEE Transactions on Power Systems.

[38]  Dazhi Yang,et al.  Kriging for NSRDB PSM version 3 satellite-derived solar irradiance , 2018, Solar Energy.

[39]  Carlos F.M. Coimbra,et al.  History and trends in solar irradiance and PV power forecasting: A preliminary assessment and review using text mining , 2018, Solar Energy.

[40]  Dazhi Yang,et al.  SolarData: An R package for easy access of publicly available solar datasets , 2018, Solar Energy.

[41]  Pierre Pinson,et al.  Online adaptive lasso estimation in vector autoregressive models for high dimensional wind power forecasting , 2019, International Journal of Forecasting.

[42]  Shinichi Inage,et al.  Development of an advection model for solar forecasting based on ground data first report: Development and verification of a fundamental model , 2017 .

[43]  S. Miller,et al.  Short-term solar irradiance forecasting via satellite/model coupling , 2017, Solar Energy.