Echo State Networks and Extreme Learning Machines: A Comparative Study on Seasonal Streamflow Series Prediction

Extreme Learning Machines (ELMs) and Echo State Networks (ESNs) represent promising alternatives in time series forecasting in view of their intrinsic trade-off between performance and mathematical tractability. Both approaches share a key feature: their supervised parameter adaptation is restricted to the output layer, the remaining synaptic weights being chosen according to a priori unsupervised schemes. This work performs a comparative investigation regarding the performances of a classic ELM and ESNs in the context of the prediction of monthly seasonal streamflow series associated with Brazilian hydroelectric plants. With respect to the ESN, two possible reservoir design approaches are tested, as well as the novel architecture of Boccato et al., which is characterized by the use a Volterra filter and PCA in the readout. Additionally, a MLP is included to establish a base for comparison. Results show the relevance of these architectures in modeling seasonal streamflow series.

[1]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[2]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[3]  José Carlos Príncipe,et al.  Water Inflow Forecasting using the Echo State Network: a Brazilian Case Study , 2007, 2007 International Joint Conference on Neural Networks.

[4]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[5]  Hugo Valadares Siqueira,et al.  Echo State Networks for Seasonal Streamflow Series Forecasting , 2012, IDEAL.

[6]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[7]  Rosangela Ballini,et al.  Top-down strategies based on adaptive fuzzy rule-based systems for daily time series forecasting , 2011 .

[8]  Fernando José Von Zuben,et al.  An extended echo state network using Volterra filtering and principal component analysis , 2012, Neural Networks.

[9]  Fernando José Von Zuben,et al.  An echo state network architecture based on volterra filtering and PCA with application to the channel equalization problem , 2011, The 2011 International Joint Conference on Neural Networks.

[10]  Fernando José Von Zuben,et al.  Improved second-order training algorithms for globally and partially recurrent neural networks , 1999, IJCNN.

[11]  Fernando J. Von Zuben,et al.  Efficient Second-Order Learning Algorithms for Discrete-Time Recurrent Neural Networks , 1999 .

[12]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[13]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[14]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[15]  俊一 甘利,et al.  A. Hyvärinen, J. Karhunen and E. Oja, Independent Component Analysis, Jhon Wiley & Sons, 2001年，504ページ． （根本幾・川勝真喜訳：独立成分分析——信号解析の新しい世界，東京電機大学出版局，2005年，532ページ．） , 2010 .

[16]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[17]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.