Spatio-Temporal Interpolated Echo State Network for Meteorological Series Prediction

Spatio-temporal series prediction has attracted increasing attention in the field of meteorology in recent years. The spatial and temporal joint effect makes predictions challenging. Most of the existing spatio-temporal prediction models are computationally complicated. To develop an accurate but easy-to-implement spatio-temporal prediction model, this paper designs a novel spatio-temporal prediction model based on echo state networks. For real-world observed meteorological data with randomness and large changes, we use a cubic spline method to bridge the gaps between the neighboring points, which results in a pleasingly smooth series. The interpolated series is later input into the spatio-temporal echo state networks, in which the spatial coefficients are computed by the elastic-net algorithm. This approach offers automatic selection and continuous shrinkage of the spatial variables. The proposed model provides an intuitive but effective approach to address the interaction of spatial and temporal effects. To demonstrate the practicality of the proposed model, we apply it to predict two real-world datasets: monthly precipitation series and daily air quality index series. Experimental results demonstrate that the proposed model achieves a normalized root-mean-square error of approximately 0.250 on both datasets. Similar results are achieved on the long short-term memory model, but the computation time of our proposed model is considerably shorter. It can be inferred that our proposed neural network model has advantages on predicting meteorological series over other models.

[1]  Zhang Yi,et al.  Recurrent Neural Networks With Auxiliary Memory Units , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Noel A. C. Cressie,et al.  Statistics for Spatial Data: Cressie/Statistics , 1993 .

[3]  Derong Liu,et al.  Echo state network-based Q-learning method for optimal battery control of offices combined with renewable energy , 2017 .

[4]  Ercan Kahya,et al.  Daily streamflow modelling using autoregressive moving average and artificial neural networks models: case study of Çoruh basin, Turkey , 2012 .

[5]  Yicong Zhou,et al.  Dynamic Parameter-Control Chaotic System , 2016, IEEE Transactions on Cybernetics.

[6]  Jan Hauke,et al.  Comparison of Values of Pearson's and Spearman's Correlation Coefficients on the Same Sets of Data , 2011 .

[7]  Michael Unser,et al.  Splines: a perfect fit for signal and image processing , 1999, IEEE Signal Process. Mag..

[8]  Amir F. Atiya,et al.  A Bias and Variance Analysis for Multistep-Ahead Time Series Forecasting , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Zoubeida Bargaoui,et al.  Comparison of two kriging interpolation methods applied to spatiotemporal rainfall , 2009 .

[10]  Eder Santana,et al.  Exploiting Spatio-Temporal Structure with Recurrent Winner-Take-All Networks , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[11]  A. Gallant,et al.  Estimating the Lyapunov Exponent of a Chaotic System with Nonparametric Regression , 1992 .

[12]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[13]  Haibo He,et al.  Spatio–Temporal Memories for Machine Learning: A Long-Term Memory Organization , 2009, IEEE Transactions on Neural Networks.

[14]  Yiannis Demiris,et al.  Spatio-Temporal Learning With the Online Finite and Infinite Echo-State Gaussian Processes , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[16]  Henry Leung,et al.  Nonlinear spatial-temporal prediction based on optimal fusion , 2006, IEEE Trans. Neural Networks.

[17]  Benjamin Schrauwen,et al.  Reservoir Computing Trends , 2012, KI - Künstliche Intelligenz.

[18]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[19]  Haibo He,et al.  Optimal Control for Unknown Discrete-Time Nonlinear Markov Jump Systems Using Adaptive Dynamic Programming , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[20]  A. Comrie,et al.  Real-Time Ozone Mapping Using a Regression-Interpolation Hybrid Approach, Applied to Tucson, Arizona , 2004, Journal of the Air & Waste Management Association.

[21]  Zhongyi Hu,et al.  PSO-MISMO Modeling Strategy for MultiStep-Ahead Time Series Prediction , 2014, IEEE Transactions on Cybernetics.

[22]  Narayanan Kumarappan,et al.  Day-Ahead Deregulated Electricity Market Price Forecasting Using Recurrent Neural Network , 2013, IEEE Systems Journal.

[23]  Min Han,et al.  Adaptive Elastic Echo State Network for Multivariate Time Series Prediction , 2016, IEEE Transactions on Cybernetics.

[24]  Min Han,et al.  Laplacian Echo State Network for Multivariate Time Series Prediction , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  C. L. Philip Chen,et al.  Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Kadri Yürekli,et al.  Application of linear stochastic models to monthly flow data of Kelkit Stream , 2005 .

[27]  Jochen J. Steil,et al.  Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning , 2007, Neural Networks.

[28]  Zhigang Zeng,et al.  Prediction Intervals for Landslide Displacement Based on Switched Neural Networks , 2016, IEEE Transactions on Reliability.

[29]  Yiannis Demiris,et al.  Echo State Gaussian Process , 2011, IEEE Transactions on Neural Networks.

[30]  Rohitash Chandra,et al.  Competition and Collaboration in Cooperative Coevolution of Elman Recurrent Neural Networks for Time-Series Prediction , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Ying Bai,et al.  On the Comparison of Trilinear, Cubic Spline, and Fuzzy Interpolation Methods in the High-Accuracy Measurements , 2010, IEEE Transactions on Fuzzy Systems.

[32]  Yuichi Motai,et al.  Kernel Association for Classification and Prediction: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Jun Wang,et al.  Model Predictive Control of Unknown Nonlinear Dynamical Systems Based on Recurrent Neural Networks , 2012, IEEE Transactions on Industrial Electronics.

[34]  Yue Joseph Wang,et al.  Nonlinear System Modeling With Random Matrices: Echo State Networks Revisited , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[35]  J. Oeppen The identification of regional forecasting models using space-time correlation , 1975 .

[36]  Zhigang Zeng,et al.  A short-term power load forecasting model based on the generalized regression neural network with decreasing step fruit fly optimization algorithm , 2017, Neurocomputing.

[37]  Dawn M. Tilbury,et al.  Multi-Step Ahead Predictions for Critical Levels in Physiological Time Series , 2016, IEEE Transactions on Cybernetics.

[38]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[39]  Na Chen,et al.  Error Analysis for Matrix Elastic-Net Regularization Algorithms , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Donald E. Brown,et al.  A new point process transition density model for space-time event prediction , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[41]  Junfei Qiao,et al.  Growing Echo-State Network With Multiple Subreservoirs , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Nan Chen,et al.  Solar irradiance forecasting using spatial-temporal covariance structures and time-forward kriging , 2013 .

[43]  Abbas Khosravi,et al.  Short-Term Load and Wind Power Forecasting Using Neural Network-Based Prediction Intervals , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Tim Li,et al.  A spatial–temporal projection model for 10–30 day rainfall forecast in South China , 2015, Climate Dynamics.

[45]  Danwei Wang,et al.  Robust Recurrent Kernel Online Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Saleh A. Wasimi,et al.  Generating and forecasting monthly flows of the Ganges river with PAR model , 2006 .

[47]  Claudio Gallicchio,et al.  Architectural and Markovian factors of echo state networks , 2011, Neural Networks.

[48]  Celia Anteneodo,et al.  Analytical results for coupled-map lattices with long-range interactions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[49]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[50]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[51]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[52]  Witold Pedrycz,et al.  Clustering Spatiotemporal Data: An Augmented Fuzzy C-Means , 2013, IEEE Transactions on Fuzzy Systems.

[53]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[54]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[55]  Lorenzo Livi,et al.  Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[56]  Stefan C. Kremer,et al.  On the computational power of Elman-style recurrent networks , 1995, IEEE Trans. Neural Networks.

[57]  Ammar Belatreche,et al.  SpikeTemp: An Enhanced Rank-Order-Based Learning Approach for Spiking Neural Networks With Adaptive Structure , 2017, IEEE Transactions on Neural Networks and Learning Systems.