Multi-objective ensembles of echo state networks and extreme learning machines for streamflow series forecasting

Abstract Streamflow series forecasting composes a fundamental step in planning electric energy production for hydroelectric plants. In Brazil, such plants produce almost 70% of the total energy. Therefore, it is of great importance to improve the quality of streamflow series forecasting by investigating state-of-the-art time series forecasting algorithms. To this end, this work proposes the development of ensembles of unorganized machines, namely Extreme Learning Machines (ELMs) and Echo State Networks (ESNs). Two primary contributions are proposed: (1) a new training logic for ESNs that enables the application of bootstrap aggregation (bagging); and (2) the employment of multi-objective optimization to select and adjust the weights of the ensemble’s base models, taking into account the trade-off between bias and variance. Experiments are conducted on streamflow series data from five real-world Brazilian hydroelectric plants, namely those in Sobradinho, Serra da Mesa, Jirau, Furnas and Agua Vermelha. The statistical results for four different prediction horizons (1, 3, 6, and 12 months ahead) indicate that the ensembles of unorganized machines achieve better results than autoregressive (AR) models in terms of the Nash–Sutcliffe model efficiency coefficient (NSE), root mean squared error (RMSE), coefficient of determination (R 2 ), and RMSE-observations standard deviation ratio (RSR). In such results, the ensembles with ESNs and the multi-objective optimization design procedure achieve the best scores.

[1]  Pascal Côté,et al.  Analysis of the effects of biases in ensemble streamflow prediction (ESP) forecasts on electricity production in hydropower reservoir management , 2019, Hydrology and Earth System Sciences.

[2]  Lior Rokach,et al.  Ensemble-based classifiers , 2010, Artificial Intelligence Review.

[3]  Ahmad Sharafati,et al.  Development of multivariate adaptive regression spline integrated with differential evolution model for streamflow simulation , 2019, Journal of Hydrology.

[4]  Gilberto Reynoso-Meza,et al.  A study of Pareto-based methods for ensemble pool generation and aggregation , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[5]  S. Prasher,et al.  Evaluating Three Hydrological Distributed Watershed Models: MIKE-SHE, APEX, SWAT , 2014 .

[6]  Gilberto Reynoso-Meza,et al.  Controller tuning using evolutionary multi-objective optimisation: Current trends and applications , 2014 .

[7]  Zaher Mundher Yaseen,et al.  Novel approach for streamflow forecasting using a hybrid ANFIS-FFA model , 2017 .

[8]  J. Wolfowitz,et al.  On a Test Whether Two Samples are from the Same Population , 1940 .

[9]  Amaury Lendasse,et al.  Methodology for long-term prediction of time series , 2007, Neurocomputing.

[10]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[11]  Nadhir Al-Ansari,et al.  Implementation of Univariate Paradigm for Streamflow Simulation Using Hybrid Data-Driven Model: Case Study in Tropical Region , 2019, IEEE Access.

[12]  Xavier Blasco Ferragud,et al.  Physical programming for preference driven evolutionary multi-objective optimization , 2014, Appl. Soft Comput..

[13]  Zaher Mundher Yaseen,et al.  Learning from Multiple Models Using Artificial Intelligence to Improve Model Prediction Accuracies: Application to River Flows , 2018, Water Resources Management.

[14]  Achille Messac,et al.  Physical programming - Effective optimization for computational design , 1996 .

[15]  Zaher Mundher Yaseen,et al.  An enhanced extreme learning machine model for river flow forecasting: State-of-the-art, practical applications in water resource engineering area and future research direction , 2019, Journal of Hydrology.

[16]  Francesca Mangili,et al.  Should We Really Use Post-Hoc Tests Based on Mean-Ranks? , 2015, J. Mach. Learn. Res..

[17]  K. Taylor.  Summarizing multiple aspects of model performance in a single diagram , 2001 .

[18]  Aranildo R. Lima,et al.  Forecasting daily streamflow using online sequential extreme learning machines , 2016 .

[19]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[20]  Xavier Blasco Ferragud,et al.  Preference driven multi-objective optimization design procedure for industrial controller tuning , 2016, Inf. Sci..

[21]  Hugo Valadares Siqueira,et al.  Echo State Networks in Seasonal Streamflow Series Prediction , 2012 .

[22]  Knut Alfredsen,et al.  Regional Statistical and Precipitation–Runoff Modelling for Ecological Applications: Prediction of Hourly Streamflow in Regulated Rivers and Ungauged Basins , 2017 .

[23]  Hugo Valadares Siqueira,et al.  Performance comparison of feedforward neural networks applied to stream flow series forecasting , 2019 .

[24]  Yaochu Jin,et al.  Classifier ensembles for image identification using multi-objective Pareto features , 2017, Neurocomputing.

[25]  Luis Samaniego,et al.  Multi-model ensemble projections of European river floods and high flows at 1.5, 2, and 3 degrees global warming , 2017 .

[26]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[27]  Diana Sarita Hamburger,et al.  Modelo Empírico Linear para Previsão de Vazão de Rios na Usina Hidrelétrica de Itaipu – Bacia do Rio Paraná , 2018, Revista Brasileira de Meteorologia.

[28]  Gregory W. Corder,et al.  Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach , 2009 .

[29]  Amanda J. C. Sharkey,et al.  Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems , 1999 .

[30]  Aranildo R. Lima,et al.  Nonlinear regression in environmental sciences using extreme learning machines: A comparative evaluation , 2015, Environ. Model. Softw..

[31]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[32]  George D. C. Cavalcanti,et al.  Dynamic classifier selection: Recent advances and perspectives , 2018, Inf. Fusion.

[33]  Leandro dos Santos Coelho,et al.  Multi-Objective Ensemble Model for Short-Term Price Forecasting in Corn Price Time Series , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[34]  Jorge Hernandez-Ambato,et al.  Multistep-ahead Streamflow and Reservoir Level Prediction Using ANNs for Production Planning in Hydroelectric Stations , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[35]  Jianzhong Zhou,et al.  Streamflow estimation by support vector machine coupled with different methods of time series decomposition in the upper reaches of Yangtze River, China , 2016, Environmental Earth Sciences.

[36]  Paulo S. G. de Mattos Neto,et al.  Correcting and combining time series forecasters , 2014, Neural Networks.

[37]  Hugo Valadares Siqueira,et al.  Performance analysis of unorganized machines in streamflow forecasting of Brazilian plants , 2018, Appl. Soft Comput..

[38]  W. Collischonn,et al.  Performance of Deterministic and Probabilistic Hydrological Forecasts for the Short-Term Optimization of a Tropical Hydropower Reservoir , 2016, Water Resources Management.

[39]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[40]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[41]  K. Sudheer,et al.  Quantification of the predictive uncertainty of artificial neural network based river flow forecast models , 2012, Stochastic Environmental Research and Risk Assessment.

[42]  Milan Stojković,et al.  A New Approach for Trend Assessment of Annual Streamflows: a Case Study of Hydropower Plants in Serbia , 2017, Water Resources Management.

[43]  Enrico Zio,et al.  Ensemble of optimized echo state networks for remaining useful life prediction , 2017, Neurocomputing.

[44]  Gilberto Reynoso-Meza,et al.  Ensemble learning by means of a multi-objective optimization design approach for dealing with imbalanced data sets , 2020, Expert Syst. Appl..

[45]  Fulei Chu,et al.  Ensemble deep learning-based fault diagnosis of rotor bearing systems , 2019, Comput. Ind..

[46]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[47]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[48]  George D. C. Cavalcanti,et al.  Hybrid intelligent system for air quality forecasting using phase adjustment , 2014, Eng. Appl. Artif. Intell..

[49]  Leandro dos Santos Coelho,et al.  Ensemble approach based on bagging, boosting and stacking for short-term prediction in agribusiness time series , 2020, Appl. Soft Comput..

[50]  Hugo Valadares Siqueira.  Unorganized machines to seasonal streamflow series forecasting , 2013, Int. J. Neural Syst..