Performance analysis of unorganized machines in streamflow forecasting of Brazilian plants

Abstract This work performs an extensive investigation about the application of unorganized machines – extreme learning machines and echo state networks – to predict monthly seasonal streamflow series, associated to three important Brazilian hydroelectric plants, for many forecasting horizons. The aforementioned models are neural network architectures which present efficient and simple training processes. Moreover, the selection of the best inputs of each model is carried out by the wrapper method, using three different evaluation criteria, and three filters, viz., those based on the partial autocorrelation function, the mutual information and the normalization of maximum relevance and minimum common redundancy method. This study also establishes a comparison between the unorganized machines and two classical models: the partial autoregressive model and the multilayer perceptron. The computational results demonstrate that the unorganized machines, especially the echo state networks, represent efficient alternatives to solve the task.

[1]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[2]  Amaury Lendasse,et al.  Methodology for long-term prediction of time series , 2007, Neurocomputing.

[3]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[4]  Guoqiang Peter Zhang,et al.  A neural network ensemble method with jittered training data for time series forecasting , 2007, Inf. Sci..

[5]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[6]  Jui-Chung Hung,et al.  A genetic algorithm approach to the spectral estimation of time series with noise and missed observations , 2008, Inf. Sci..

[7]  Stefan J. Kiebel,et al.  Re-visiting the echo state property , 2012, Neural Networks.

[8]  Hugo Valadares Siqueira,et al.  Echo State Networks and Extreme Learning Machines: A Comparative Study on Seasonal Streamflow Series Prediction , 2012, ICONIP.

[9]  Fernando José Von Zuben,et al.  An extended echo state network using Volterra filtering and principal component analysis , 2012, Neural Networks.

[10]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[11]  Johan A. K. Suykens,et al.  Improved Long-Term Temperature Prediction by Chaining of Neural Networks , 2001, Int. J. Neural Syst..

[12]  Benjamin Schrauwen,et al.  Memory in linear recurrent neural networks in continuous time , 2010, Neural Networks.

[13]  Slobodan P. Simonovic,et al.  Short term streamflow forecasting using artificial neural networks , 1998 .

[14]  Hugo Valadares Siqueira,et al.  Echo State Networks for Seasonal Streamflow Series Forecasting , 2012, IDEAL.

[15]  J. M. Damázio,et al.  The use of PAR(p) model in the stochastic dual dynamic programming optimization scheme used in the operation planning of the Brazilian hydropower system , 2005, 2004 International Conference on Probabilistic Methods Applied to Power Systems.

[16]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[17]  Nan Liu,et al.  Voting based extreme learning machine , 2012, Inf. Sci..

[18]  Fernando José Von Zuben,et al.  Data partition and variable selection for time series prediction using wrappers , 2006, IJCNN.

[19]  Laurent Larger,et al.  Stochastic Nonlinear Time Series Forecasting Using Time-Delay Reservoir Computers: Performance and Universality , 2013, Neural Networks.

[20]  Li Li,et al.  Maximum relevance minimum common redundancy feature selection for nonlinear data , 2017, Inf. Sci..

[21]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[22]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[23]  Song Li,et al.  Short-term load forecasting by wavelet transform and evolutionary extreme learning machine , 2015 .

[24]  Paulo S. G. de Mattos Neto,et al.  Correcting and combining time series forecasters , 2014, Neural Networks.

[25]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  S. Soares,et al.  Streamflow forecasting using neural networks and fuzzy clustering techniques , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[27]  Ashu Jain,et al.  Hybrid neural network models for hydrologic time series forecasting , 2007, Appl. Soft Comput..

[28]  H. Akaike.  A new look at the statistical model identification , 1974 .

[29]  Florentino Fernández Riverola,et al.  A hybrid artificial intelligence model for river flow forecasting , 2013, Appl. Soft Comput..

[30]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[31]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[32]  E. Oja,et al.  Independent Component Analysis , 2013 .

[33]  Chuanhou Gao,et al.  A comparative analysis of support vector machines and extreme learning machines , 2012, Neural Networks.

[34]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[35]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[36]  Rosangela Ballini,et al.  Top-down strategies based on adaptive fuzzy rule-based systems for daily time series forecasting , 2011 .

[37]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[38]  Hans-Georg Zimmermann,et al.  Recurrent Neural Networks are Universal approximators , 2007, Int. J. Neural Syst..

[39]  Lambros Ekonomou,et al.  Electricity demand load forecasting of the Hellenic power system using an ARMA model , 2010 .

[40]  Fernando José Von Zuben,et al.  Improved second-order training algorithms for globally and partially recurrent neural networks , 1999, IJCNN.

[41]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[42]  Amaury Lendasse,et al.  Long-term time series prediction using OP-ELM , 2014, Neural Networks.

[43]  Pankaj Singh,et al.  Suitability of different neural networks in daily flow forecasting , 2007, Appl. Soft Comput..

[44]  Ping Li,et al.  A review on Gaussian Process Latent Variable Models , 2016, CAAI Trans. Intell. Technol..

[45]  Yi Zhang,et al.  Application of a hybrid quantized Elman neural network in short-term load forecasting , 2014 .

[46]  José Carlos Príncipe,et al.  Water Inflow Forecasting using the Echo State Network: a Brazilian Case Study , 2007, 2007 International Joint Conference on Neural Networks.

[47]  Fernando Gomide,et al.  A Seasonal Streamflow Forecasting Model Using Neurofuzzy Network , 2000 .

[48]  Hugo Valadares Siqueira,et al.  Echo State Networks in Seasonal Streamflow Series Prediction , 2012 .

[49]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[50]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[51]  Hugo Valadares Siqueira.  Unorganized machines to seasonal streamflow series forecasting , 2013, Int. J. Neural Syst..

[52]  Levy Boccato,et al.  Unorganized Machines: From Turing's Ideas to Modern Connectionist Approaches , 2011, Int. J. Nat. Comput. Res..

[53]  Toly Chen,et al.  Long-term load forecasting by a collaborative fuzzy-neural approach , 2012 .

[54]  A. I. McLeod.  DIAGNOSTIC CHECKING OF PERIODIC AUTOREGRESSION MODELS WITH APPLICATION , 1994 .