Data Analytics for Renewable Energy Integration: Informing the Generation and Distribution of Renewable Energy

Solar energy forecasting represents a key issue in order to efficiently manage the supply-demand balance and promote an effective renewable energy integration. In this regard, an accurate solar energy forecast is of utmoss importance for avoiding large voltage variations into the electricity network and providing the system with mechanisms for managing the produced energy in an optimal way. This paper presents a novel solar energy forecasting and optimization approach called SUNSET which efficiently determines the optimal energy management for the next 24 h in terms of: self-consumption, energy purchase and battery energy storage for later consumption. The proposed SUNSET approach has been tested in a real solar PV system plant installed in Zamudio (Spain) and compared towards a Real-Time (RT) strategy in terms of price and energy savings obtaining attractive results.

[1]  Jin Jiang,et al.  Modeling, Prediction, and Experimental Validations of Power Peaks of PV Arrays Under Partial Shading Conditions , 2014, IEEE Transactions on Sustainable Energy.

[2]  Taehoon Hong,et al.  An estimation model for determining the annual energy cost budget in educational facilities using SARIMA (seasonal autoregressive integrated moving average) and ANN (artificial neural network) , 2014 .

[3]  Balázs Kégl,et al.  Ranking by calibrated AdaBoost , 2010, Yahoo! Learning to Rank Challenge.

[4]  Efstratios I. Batzelis,et al.  Energy models for photovoltaic systems under partial shading conditions: a comprehensive review , 2015 .

[5]  P. F. van den Oosterkamp,et al.  The role of DSOs in a Smart Grid environment , 2014 .

[6]  Sergejus Martinenas,et al.  Scheduling of domestic water heater power demand for maximizing PV self-consumption using model predictive control , 2013, IEEE PES ISGT Europe 2013.

[7]  Philippe Lauret,et al.  Probabilistic forecasting of the solar irradiance with recursive ARMA and GARCH models , 2016 .

[8]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[9]  Bernhard Sick,et al.  Online Segmentation of Time Series Based on Polynomial Least-Squares Approximations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Snezhana Georgieva Gocheva-Ilieva,et al.  Time series analysis and forecasting for air pollution in small urban area: an SARIMA and factor analysis approach , 2014, Stochastic Environmental Research and Risk Assessment.

[11]  Jing Deng,et al.  Hybrid Probabilistic Wind Power Forecasting Using Temporally Local Gaussian Process , 2016, IEEE Transactions on Sustainable Energy.

[12]  James Merrick.  On representation of temporal variability in electricity capacity planning models , 2016 .

[13]  Justin Heinermann,et al.  Wind Power Prediction with Machine Learning Ensembles , 2016 .

[14]  M. Diagne,et al.  Review of solar irradiance forecasting methods and a proposition for small-scale insular grids , 2013 .

[15]  Claudio M. Rocco Sanseverino.  Singular spectrum analysis and forecasting of failure time series , 2013, Reliab. Eng. Syst. Saf..

[16]  G. Kerber.  Aufnahmefähigkeit von Niederspannungsverteilnetzen für die Einspeisung aus Photovoltaikkleinanlagen , 2011 .

[17]  Widen Tabakoff,et al.  High-temperature erosion resistance of coatings for use in turbomachinery , 1995 .

[18]  Chris Deline,et al.  A simplified model of uniform shading in large photovoltaic arrays , 2013 .

[19]  W. Woon,et al.  Artificial Neural Network-based electricity price forecasting for smart grid deployment , 2012, 2012 International Conference on Computer Systems and Industrial Informatics.

[20]  Jun Wang,et al.  Generalizing DTW to the multi-dimensional case requires an adaptive approach , 2016, Data Mining and Knowledge Discovery.

[21]  Nikos D. Hatziargyriou,et al.  A pattern recognition methodology for evaluation of load profiles and typical days of large electricity customers , 2008 .

[22]  Xiaoming Jin,et al.  Distribution Discovery: Local Analysis of Temporal Rules , 2002, PAKDD.

[23]  Irena Koprinska,et al.  Univariate and multivariate methods for very short-term solar photovoltaic power forecasting , 2016 .

[24]  Hui Liu,et al.  Comparison of two new ARIMA-ANN and ARIMA-Kalman hybrid methods for wind speed prediction , 2012 .

[25]  John Bjørnar Bremnes,et al.  Probabilistic wind power forecasts using local quantile regression , 2004 .

[26]  Vladimiro Miranda,et al.  Probabilistic solar power forecasting in smart grids using distributed information , 2015 .

[27]  Álvaro Gomes,et al.  A multi-objective genetic approach to domestic load scheduling in an energy management system , 2014 .

[28]  F. Almonacid,et al.  A simple accurate model for the calculation of shading power losses in photovoltaic generators , 2013 .

[29]  Efstratios I. Batzelis,et al.  An Explicit PV String Model Based on the Lambert $W$ Function and Simplified MPP Expressions for Operation Under Partial Shading , 2014, IEEE Transactions on Sustainable Energy.

[30]  W. Rivera,et al.  Short term wind speed forecasting in La Venta, Oaxaca, México, using artificial neural networks , 2009 .

[31]  Wei Lee Woon,et al.  Machine Learning Techniques for Supporting Renewable Energy Generation and Integration: A Survey , 2014, DARE.

[32]  Milind Tambe,et al.  Coordinating occupant behavior for building energy and comfort management using multi-agent systems , 2012 .

[33]  Edwin V. Bonilla,et al.  Generic Inference in Latent Gaussian Process Models , 2016, J. Mach. Learn. Res..

[34]  Tim N. Palmer,et al.  Ensemble forecasting , 2008, J. Comput. Phys..

[35]  N. D. Hatziargyriou,et al.  Probabilistic Wind Power Forecasting Using Radial Basis Function Neural Networks , 2012, IEEE Transactions on Power Systems.

[36]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[37]  Eamonn Keogh,et al.  On the effect of endpoints on dynamic time warping , 2016 .

[38]  Heikki Mannila,et al.  Rule Discovery from Time Series , 1998, KDD.

[39]  Wei Tian,et al.  A Hybrid Method for Short-Term Wind Speed Forecasting , 2017 .

[40]  Toni Giorgino,et al.  Matching incomplete time series with dynamic time warping: an algorithm and an application to post-stroke rehabilitation , 2009, Artif. Intell. Medicine.

[41]  Richard T. Watson,et al.  Ten questions concerning integrating smart buildings into the smart grid , 2016 .

[42]  Miguel-Ángel Manso-Callejo,et al.  Forecasting short-term solar irradiance based on artificial neural networks and data from neighboring meteorological stations , 2016 .

[43]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[44]  F. Nielsen,et al.  The Bell Curve: Intelligence and Class Structure in American Life. , 1995 .

[45]  I. G. Kamphuis,et al.  Grote concentraties warmtepompen in een woonwijk en gevolgen elektriciteitsnetwerk , 2010 .

[46]  Wesley W. Chu,et al.  Discovering and Matching Elastic Rules from Sequence Databases , 2000, Fundam. Informaticae.

[47]  R. Inman,et al.  Solar forecasting methods for renewable energy integration , 2013 .

[48]  Jitender S. Deogun,et al.  Discovering Sequential Association Rules with Constraints and Time Lags in Multiple Sequences , 2002, ISMIS.

[49]  Hsiao-Dong Chiang,et al.  A High-Accuracy Wind Power Forecasting Model , 2017, IEEE Transactions on Power Systems.

[50]  Nuno Constantino Castro,et al.  Time Series Data Mining , 2009, Encyclopedia of Database Systems.

[51]  Anastasios G. Bakirtzis.  A probabilistic method for the evaluation of the reliability of stand alone wind energy systems , 1992 .

[52]  David Mease,et al.  Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers , 2015, J. Mach. Learn. Res..

[53]  Laxman Poudel,et al.  Impact of Sand on Hydraulic Turbine Material: A Case Study of Roshi Khola, Nepal , 2012 .

[54]  Dazhi Yang,et al.  Very short term irradiance forecasting using the lasso , 2015 .

[55]  Gregor Giebel,et al.  Wind Power Prediction using Ensembles , 2005 .

[56]  H. Levene.  Robust tests for equality of variances , 1961 .

[57]  José R. Dorronsoro,et al.  Deep Neural Networks for Wind and Solar Energy Prediction , 2017, Neural Processing Letters.

[58]  Dimitrios D. Thomakos,et al.  A review on singular spectrum analysis for economic and financial time series , 2010 .

[59]  Gianluca Bontempi,et al.  Machine Learning Strategies for Time Series Forecasting , 2012, eBISS.

[60]  A. Testa,et al.  Very short-term probabilistic wind power forecasting based on Markov chain models , 2010, 2010 IEEE 11th International Conference on Probabilistic Methods Applied to Power Systems.

[61]  Tansu Filik,et al.  Wind Speed Prediction Using Artificial Neural Networks Based on Multiple Local Measurements in Eskisehir , 2017 .

[62]  Fang Zhao,et al.  A Fast Classification Algorithm for Big Data Based on KNN , 2013 .

[63]  Efstratios I. Batzelis,et al.  Analysis of Local MPPs on the P-V Curve of a Partially Shaded PV String , 2014 .

[64]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[65]  Eamonn J. Keogh,et al.  Discovery of Meaningful Rules in Time Series , 2015, KDD.

[66]  Marko Topič,et al.  Self-shading losses of fixed free-standing PV arrays , 2011 .

[67]  Chang Hoi Kim,et al.  A Study of the Growth of Single-Phase Mg0.5Zn0.5O Films for UV LED , 2014 .

[68]  Peter Lund,et al.  Review of energy system flexibility measures to enable high levels of variable renewable electricity , 2015 .

[69]  Guy P. Nason,et al.  Should we sample a time series more frequently?: decision support via multirate spectrum estimation , 2017 .

[70]  J. J. G. de la Rosa,et al.  Comparison of Models for Wind Speed Forecasting , 2009 .

[71]  Tetsuo Sasaki,et al.  Areal Solar Irradiance Estimated by Sparsely Distributed Observations of Solar Radiation , 2016, IEEE Transactions on Power Systems.

[72]  Donghui Zhang,et al.  Online event-driven subsequence matching over financial data streams , 2004, SIGMOD '04.

[73]  Chiara Delmastro,et al.  Generalizable occupant-driven optimization model for domestic hot water production in NZEB , 2016 .

[74]  Peter L. Bartlett,et al.  Boosting Algorithms as Gradient Descent , 1999, NIPS.

[75]  E. F. Tobin,et al.  Comparison of liquid impingement results from whirling arm and water-jet rain erosion test facilities , 2011 .

[76]  Ignacio J. Turias,et al.  Hybrid approaches based on SARIMA and artificial neural networks for inspection time series forecasting , 2014 .

[77]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[78]  Pinar Karagoz,et al.  A Novel Wind Power Forecast Model: Statistical Hybrid Wind Power Forecast Technique (SHWIP) , 2015, IEEE Transactions on Industrial Informatics.

[79]  Claudio Moraga,et al.  Artificial neural networks in time series forecasting: a comparative analysis , 2002, Kybernetika.

[80]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[81]  Bernhard Sick,et al.  Fast Feature Extraction for Time Series Analysis Using Least-Squares Approximations with Orthogonal Basis Functions , 2015, 2015 22nd International Symposium on Temporal Representation and Reasoning (TIME).

[82]  T. Soubdhan,et al.  A benchmarking of machine learning techniques for solar radiation forecasting in an insular context , 2015 .

[83]  J. A. Ferreira,et al.  Singular spectrum analysis and forecasting of hydrological time series , 2006 .

[84]  Mrs. Deepali Kishor Jadhav.  Big Data: The New Challenges in Data Mining , 2013 .

[85]  Christos Kaidis.  Wind Turbine Reliability Prediction : A Scada Data Processing a Reliability Estimation Tool , 2014 .

[86]  Efstratios I. Batzelis,et al.  A MPPT Algorithm for Partial Shading Conditions Employing Curve Fitting , 2016 .

[87]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[88]  Torbjorn Thiringer,et al.  ARIMA-Based Frequency-Decomposed Modeling of Wind Speed Time Series , 2016, IEEE Transactions on Power Systems.

[89]  Paul Fleming,et al.  Use of SCADA Data for Failure Detection in Wind Turbines , 2011 .

[90]  Anoop Prakash Verma.  Performance monitoring of wind turbines: A data-mining approach , 2012 .

[91]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[92]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[93]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[94]  Kuan-Yu Chen,et al.  A hybrid SARIMA and support vector machines in forecasting the production values of the machinery industry in Taiwan , 2007, Expert Syst. Appl..

[95]  Henrik Madsen,et al.  Using quantile regression to extend an existing wind power forecasting system with probabilistic forecasts , 2006 .

[96]  Iain Staffell,et al.  Divide and Conquer? ${k}$-Means Clustering of Demand Data Allows Rapid and Accurate Simulations of the British Electricity System , 2014, IEEE Transactions on Engineering Management.

[97]  Soteris A. Kalogirou,et al.  Machine learning methods for solar radiation forecasting: A review , 2017 .

[98]  Jessica Lin,et al.  Finding Motifs in Time Series , 2002, KDD 2002.

[99]  Oliver Kramer,et al.  Enhanced SVR ensembles for wind power prediction , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[100]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[101]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[102]  Laurent Georges,et al.  Advanced control of heat pumps for improved flexibility of Net-ZEB towards the grid , 2014 .

[103]  Efstratios I. Batzelis,et al.  Partial Shading Analysis of Multistring PV Arrays and Derivation of Simplified MPP Expressions , 2015, IEEE Transactions on Sustainable Energy.

[104]  Mihai Anitescu,et al.  Data-driven model for solar irradiation based on satellite observations , 2014 .

[105]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[106]  Mikko Kolehmainen,et al.  Feature-Based Clustering for Electricity Use Time Series Data , 2009, ICANNGA.

[107]  Michael S. Selig,et al.  Simulation of Damage for Wind Turbine Blades Due to Airborne Particles , 2015 .

[108]  Zeyar Aung,et al.  Ensemble Learning Approach for Probabilistic Forecasting of Solar Power Generation , 2016 .

[109]  N. Bigdeli,et al.  Data analysis and short term load forecasting in Iran electricity market using singular spectral analysis (SSA) , 2011 .

[110]  Pierre Pinson,et al.  Very Short-Term Nonparametric Probabilistic Forecasting of Renewable Energy Generation— With Application to Solar Energy , 2016, IEEE Transactions on Power Systems.

[111]  Long Bao Le,et al.  Joint Optimization of Electric Vehicle and Home Energy Scheduling Considering User Comfort Preference , 2014, IEEE Transactions on Smart Grid.

[112]  Fabio Polonara,et al.  Domestic demand-side management (DSM): Role of heat pumps and thermal energy storage (TES) systems , 2013 .

[113]  Christian P. Robert,et al.  Statistics for Spatio-Temporal Data , 2014 .

[114]  Zeyar Aung,et al.  Probabilistic Forecasting of Solar Power: An Ensemble Learning Approach , 2017, KES-IDT.

[115]  Abraham K. Ishihara,et al.  Neural network estimation of photovoltaic I–V curves under partially shaded conditions , 2011, The 2011 International Joint Conference on Neural Networks.

[116]  Peng Kou,et al.  Stochastic predictive control of battery energy storage for wind farm dispatching: Using probabilistic wind power forecasts , 2015 .

[117]  Senén Barro,et al.  Do we need hundreds of classifiers to solve real world classification problems? , 2014, J. Mach. Learn. Res..

[118]  John Boland,et al.  Spatial-temporal forecasting of solar radiation , 2015 .

[119]  G.N. Kariniotakis,et al.  Probabilistic Short-term Wind Power Forecasting for the Optimal Management of Wind Generation , 2007, 2007 IEEE Lausanne Power Tech.

[120]  Nan Chen,et al.  Solar irradiance forecasting using spatial-temporal covariance structures and time-forward kriging , 2013 .

[121]  L. D. Monache,et al.  An application of the ECMWF Ensemble Prediction System for short-term solar power forecasting , 2016 .

[122]  Michael S. Selig,et al.  Simulation of Damage Progression on Wind Turbine Blades Subject to Particle Erosion , 2016 .

[123]  Ning Lu,et al.  Appliance Commitment for Household Load Scheduling , 2011, IEEE Transactions on Smart Grid.

[124]  Anthony R. Florita,et al.  Value of Improved Short-Term Wind Power Forecasting , 2015 .

[125]  J. B. Bremnes.  A comparison of a few statistical models for making quantile wind power forecasts , 2006 .

[126]  Dazhi Yang,et al.  Very short-term irradiance forecasting at unobserved locations using spatio-temporal kriging , 2015 .

[127]  Sherin M. Moussa,et al.  The Evolution of Data Mining Techniques to Big Data Analytics: An Extensive Study with Application to Renewable Energy Data Analytics , 2016 .

[128]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[129]  D. Fadare.  The application of artificial neural networks to mapping of wind speed profile for energy application in Nigeria , 2010 .

[130]  Xiaoli Li,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. 1 Classification of Energy Consumption in Buildings with Outlier Detection , 2022 .

[131]  Prashant J. Shenoy,et al.  Integrating Energy Storage in Electricity Distribution Networks , 2015, e-Energy.

[132]  Eamonn J. Keogh,et al.  Clustering of time-series subsequences is meaningless: implications for previous and future research , 2004, Knowledge and Information Systems.

[133]  Christopher A. Walford,et al.  Wind Turbine Reliability: Understanding and Minimizing Wind Turbine Operation and Maintenance Costs , 2006 .

[134]  Álvaro Alonso,et al.  Random Forests and Gradient Boosting for Wind Energy Prediction , 2015, HAIS.

[135]  Roger Ray Hill,et al.  Wind turbine reliability : a database and analysis approach. , 2008 .

[136]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[137]  Glenn Platt,et al.  Machine learning for solar irradiance forecasting of photovoltaic system , 2016 .

[138]  John Boland,et al.  Short term solar radiation forecasting: Island versus continental sites , 2016 .

[139]  Alessandro Agnetis,et al.  Load Scheduling for Household Energy Consumption Optimization , 2013, IEEE Transactions on Smart Grid.

[140]  Marco Levorato,et al.  Residential Demand Response Using Reinforcement Learning , 2010, 2010 First IEEE International Conference on Smart Grid Communications.

[141]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[142]  Magnus Lie Hetland,et al.  Temporal Rule Discovery using Genetic Programming and Specialized Hardware , 2004 .

[143]  P. Guttorp,et al.  Nonparametric Estimation of Nonstationary Spatial Covariance Structure , 1992 .

[144]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[145]  Neil D. Lawrence,et al.  Gaussian Processes for Big Data , 2013, UAI.

[146]  Thomas Ackermann,et al.  Wind Power in Power Systems , 2005 .

[147]  David B. Richardson,et al.  Electric vehicles and the electric grid: A review of modeling approaches, Impacts, and renewable energy integration , 2013 .

[148]  Ponnuthurai Nagaratnam Suganthan,et al.  Ensemble methods for wind and solar power forecasting—A state-of-the-art review , 2015 .

[149]  Peter A. Flach,et al.  Cost-sensitive boosting algorithms: Do we really need them? , 2016, Machine Learning.

[150]  Dilek Küçük,et al.  Enhanced Nationwide Wind-Electric Power Monitoring and Forecast System , 2014, IEEE Transactions on Industrial Informatics.

[151]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[152]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[153]  A. Woyte,et al.  Rule-based demand-side management of domestic hot water production with heat pumps in zero energy neighbourhoods , 2014 .

[154]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[155]  Mehdi Khashei,et al.  An artificial neural network (p, d, q) model for timeseries forecasting , 2010, Expert Syst. Appl..

[156]  Chih-Jen Lin,et al.  Simple Probabilistic Predictions for Support Vector Regression , 2004 .

[157]  Howard W Mielke,et al.  The Fisher-Pitman Permutation Test: An Attractive Alternative to the F Test , 2002, Psychological reports.

[158]  Erik Delarue,et al.  Selecting Representative Days for Capturing the Implications of Integrating Intermittent Renewables in Generation Expansion Planning Problems , 2017, IEEE Transactions on Power Systems.

[159]  A. Cronin,et al.  Intra-hour forecasts of solar power production using measurements from a network of irradiance sensors , 2013 .

[160]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[161]  Alistair B. Sproul,et al.  Photovoltaic (PV) performance modelling in the absence of onsite measured plane of array irradiance (POA) and module temperature , 2016 .

[162]  A. Shabri,et al.  A comparison of time series forecasting using support vector machine and artificial neural network model , 2010 .