Parallel deep prediction with covariance intersection fusion on non-stationary time series

Abstract The prediction of time series can help the advanced estimation and management for scientific decision-making in many fields. The non-stationary trend and complex noises make the time series hard to predict. To utilize the learning ability of the deep neural networks, a novel hybrid prediction method is proposed combining data decomposition and parallel deep network. First, the original time series data are divided into multiple groups with a self-defined method for the following parallel networks. Then, each group of data is decomposed into the period, trend, and residual components for the precise feature extraction. Each component is separately trained with a deep network, and the networks are selected to form the parallel prediction model. Finally, the covariance intersection fusion method is introduced to integrate the multiple results based on the estimation of error level. The proposed method is tested on the meteorological data and air pollutant data in Beijing. The method achieves optimal results than the single deep network and traditional integration method. The parallel deep prediction model with the covariance intersection can extract the time series features more integrally and subtly. And it can effectively predict the non-stationary time series with complex noises.

[1]  Jie Ding,et al.  Particle filtering based parameter estimation for systems with output-error type model structures , 2019, J. Frankl. Inst..

[2]  Hao Zhou,et al.  A gated recurrent unit neural networks based wind speed error correction model for short-term wind power forecasting , 2019, Neurocomputing.

[3]  Tingli Su,et al.  Integrated Predictor Based on Decomposition Mechanism for PM2.5 Long-Term Prediction , 2019, Applied Sciences.

[4]  N. Amjady.  Day-ahead price forecasting of electricity markets by a new fuzzy neural network , 2006, IEEE Transactions on Power Systems.

[5]  Baihai Zhang,et al.  A Neuron-Based Kalman Filter with Nonlinear Autoregressive Model , 2020, Sensors.

[6]  Irma J. Terpenning,et al.  STL : A Seasonal-Trend Decomposition Procedure Based on Loess , 1990 .

[7]  Jesús Carlos Pedraza Ortega,et al.  Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks , 2020, Earth Science Informatics.

[8]  Qiang Liu,et al.  Financial time series prediction using ℓ2, 1RF-ELM , 2018, Neurocomputing.

[9]  Dongxiao Niu,et al.  Wind Power Short-Term Forecasting Hybrid Model Based on CEEMD-SE Method , 2019, Processes.

[10]  Yuanxi Yang,et al.  Analysis of seasonal signals and long-term trends in the height time series of IGS sites in China , 2016, Science China Earth Sciences.

[11]  Yuan Zhao,et al.  A new prediction method based on VMD-PRBF-ARMA-E model considering wind speed characteristic , 2020 .

[12]  Weide Li,et al.  Effective passenger flow forecasting using STL and ESN based on two improvement strategies , 2019, Neurocomputing.

[13]  Chuntian Cheng,et al.  Using support vector machines for long-term discharge prediction , 2006 .

[14]  Tingli Su,et al.  Compound Autoregressive Network for Prediction of Multivariate Time Series , 2019, Complex..

[15]  Jie Li,et al.  Wavelet Decomposition and Convolutional LSTM Networks Based Improved Deep Learning Model for Solar Irradiance Forecasting , 2018, Applied Sciences.

[16]  Ting-Li Su,et al.  Hybrid Deep Learning Predictor for Smart Agriculture Sensing Based on Empirical Mode Decomposition and Gated Recurrent Unit Group Model , 2020, Sensors.

[17]  Yong Liu,et al.  Six-decade temporal change and seasonal decomposition of climate variables in Lake Dianchi watershed (China): stable trend or abrupt shift? , 2014, Theoretical and Applied Climatology.

[18]  J. Torres,et al.  Forecast of hourly average wind speed with ARMA models in Navarre (Spain) , 2005 .

[19]  Jian Ma,et al.  A Deep Neural Network Model for Short-Term Load Forecast Based on Long Short-Term Memory Network and Convolutional Neural Network , 2018, Energies.

[20]  Jürgen Schmidhuber,et al.  LSTM can Solve Hard Long Time Lag Problems , 1996, NIPS.

[21]  Chao Wen,et al.  A deep learning approach for multi-attribute data: A study of train delay prediction in railway systems , 2020, Inf. Sci..

[22]  Gang Zhang,et al.  A Hybrid Forecasting Method for Solar Output Power Based on Variational Mode Decomposition, Deep Belief Networks and Auto-Regressive Moving Average , 2018, Applied Sciences.

[23]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[24]  Xiao-yi Wang,et al.  Dynamic Correlation Analysis Method of Air Pollutants in Spatio-Temporal Analysis , 2020, International journal of environmental research and public health.

[25]  Tingli Su,et al.  Deep Hybrid Model Based on EMD with Classification by Frequency Characteristics for Long-Term Air Quality Prediction , 2020, Mathematics.

[26]  Su Yang,et al.  An echo state network architecture based on quantum logic gate and its optimization , 2020, Neurocomputing.

[27]  Jun Qiu,et al.  Monthly Streamflow Forecasting Using EEMD-Lasso-DBN Method Based on Multi-Scale Predictors Selection , 2018, Water.

[28]  Jie Yu,et al.  Spatio-temporal predictions of SST time series in China’s offshore waters using a regional convolution long short-term memory (RC-LSTM) network , 2020 .

[29]  Ting Cui,et al.  Joint Multi-innovation Recursive Extended Least Squares Parameter and State Estimation for a Class of State-space Systems , 2020 .

[30]  Qian Sun,et al.  Spatio-Temporal Prediction for the Monitoring-Blind Area of Industrial Atmosphere Based on the Fusion Network , 2019, International journal of environmental research and public health.

[31]  Yulei Rao,et al.  A deep learning framework for financial time series using stacked autoencoders and long-short term memory , 2017, PloS one.

[32]  Min Zuo,et al.  CropDeep: The Crop Vision Dataset for Deep-Learning-Based Classification and Detection in Precision Agriculture , 2019, Sensors.

[33]  Li Tang,et al.  Predicting the direction of stock markets using optimized neural networks with Google Trends , 2018, Neurocomputing.

[34]  Yu-Rong Zeng,et al.  Forecasting Tourist Arrivals via Random Forest and Long Short-term Memory , 2020, Cognitive Computation.

[35]  James Cadzow,et al.  ARMA Time Series Modeling: an Effective Method , 1983, IEEE Transactions on Aerospace and Electronic Systems.

[36]  A. Luati,et al.  Global and local statistical properties of fixed-length nonparametric smoothers , 2002 .

[37]  J. Contreras,et al.  ARIMA models to predict next-day electricity prices , 2002 .

[38]  Chao Chen,et al.  A novel axle temperature forecasting method based on decomposition, reinforcement learning optimization and neural network , 2020, Adv. Eng. Informatics.

[39]  F. Ding.  Coupled-least-squares identification for multivariable systems , 2013 .

[40]  Jorge Caiado,et al.  A GARCH-based method for clustering of financial time series: International stock markets evidence , 2007 .

[41]  Thomas Fischer,et al.  Deep learning with long short-term memory networks for financial market predictions , 2017, Eur. J. Oper. Res..

[42]  Xiangkui Wan,et al.  Two-stage Gradient-based Iterative Estimation Methods for Controlled Autoregressive Systems Using the Measurement Data , 2020, International Journal of Control, Automation and Systems.

[43]  Li Bin.  Chaotic Time Series Prediction Based on ELM Learning Algorithm , 2011 .

[44]  Wenhu Tang,et al.  Deep Learning for Daily Peak Load Forecasting–A Novel Gated Recurrent Neural Network Combining Dynamic Time Warping , 2019, IEEE Access.

[45]  Terrence L. Chambers,et al.  Hour-Ahead Solar Irradiance Forecasting Using Multivariate Gated Recurrent Units , 2019, Energies.

[46]  Baihua Xiao,et al.  Short-term cloud coverage prediction using the ARIMA time series model , 2018 .

[47]  Yong Xu,et al.  VS-GRU: A Variable Sensitive Gated Recurrent Neural Network for Multivariate Time Series with Massive Missing Values , 2019, Applied Sciences.

[48]  Xiao Jing,et al.  Short-Term Electricity Load Forecasting Model Based on EMD-GRU with Feature Selection , 2019, Energies.

[49]  Yining Wang,et al.  The Forecasting of PM2.5 Using a Hybrid Model Based on Wavelet Transform and an Improved Deep Learning Algorithm , 2019, IEEE Access.

[50]  Ilkay Oksuz,et al.  Electricity Price Forecasting Using Recurrent Neural Networks , 2018 .

[51]  Yan Chen,et al.  A Novel Dual-Scale Deep Belief Network Method for Daily Urban Water Demand Forecasting , 2018 .

[52]  Tingli Su,et al.  Deep Learning Predictor for Sustainable Precision Agriculture Based on Internet of Things System , 2020, Sustainability.

[53]  Junfei Qiao,et al.  Dynamical regularized echo state network for time series prediction , 2019, Neural Computing and Applications.

[54]  Henrik Madsen,et al.  Wind Power Forecasting Based on Echo State Networks and Long Short-Term Memory , 2018 .

[55]  Marco van Akkeren,et al.  A GARCH forecasting model to predict day-ahead electricity prices , 2005, IEEE Transactions on Power Systems.