Modeling and Analysis of Data-Driven Systems through Computational Neuroscience Wavelet-Deep Optimized Model for Nonlinear Multicomponent Data Forecasting

Complex time series data exists widely in actual systems, and its forecasting has great practical significance. Simultaneously, the classical linear model cannot obtain satisfactory performance due to nonlinearity and multicomponent characteristics. Based on the data-driven mechanism, this paper proposes a deep learning method coupled with Bayesian optimization based on wavelet decomposition to model the time series data and forecasting its trend. Firstly, the data is decomposed by wavelet transform to reduce the complexity of the time series data. The Gated Recurrent Unit (GRU) network is trained as a submodel for each decomposition component. The hyperparameters of wavelet decomposition and each submodel are optimized with Bayesian sequence model-based optimization (SMBO) to develop the modeling accuracy. Finally, the results of all submodels are added to obtain forecasting results. The PM2.5 data collected by the US Air Quality Monitoring Station is used for experiments. By comparing with other networks, it can be found that the proposed method outperforms well in the multisteps forecasting task for the complex time series.

[1]  Yi-Ping Phoebe Chen,et al.  Hybrid deep learning and empirical mode decomposition model for time series applications , 2019, Expert Syst. Appl..

[2]  Ling Xu,et al.  Recursive parameter estimation methods and convergence analysis for a special class of nonlinear systems , 2019, International Journal of Robust and Nonlinear Control.

[3]  Yong Cheng,et al.  Hybrid algorithm for short-term forecasting of PM2.5 in China , 2019, Atmospheric Environment.

[4]  Xiao Zhang,et al.  Adaptive parameter estimation for a general dynamical system with unknown states , 2020, International Journal of Robust and Nonlinear Control.

[5]  Adil M. Bagirov,et al.  Prediction of monthly rainfall in Victoria, Australia: Clusterwise linear regression approach , 2017 .

[6]  Ioannis P. Panapakidis,et al.  Day-ahead natural gas demand forecasting based on the combination of wavelet transform and ANFIS/genetic algorithm/neural network model , 2017 .

[7]  J. Le Caillec.  Hypothesis Testing for Nonlinearity Detection Based on an MA Model , 2008, IEEE Transactions on Signal Processing.

[8]  Kiho Jeong,et al.  Recurrent support vector regression for a non-linear ARMA model with applications to forecasting financial returns , 2015, Comput. Stat..

[9]  Meihang Li,et al.  Maximum Likelihood Least Squares Based Iterative Estimation for a Class of Bilinear Systems Using the Data Filtering Technique , 2020 .

[10]  Qinyao Liu,et al.  Recursive coupled projection algorithms for multivariable output-error-like systems with coloured noises , 2020, IET Signal Process..

[11]  F. Ding,et al.  Recursive parameter estimation and its convergence for bilinear systems , 2020, IET Control Theory & Applications.

[12]  R. Vinayakumar,et al.  DeepAirNet: Applying Recurrent Networks for Air Quality Prediction , 2018 .

[13]  Feng Ding,et al.  Parameter estimation algorithms for dynamical response signals based on the multi-innovation theory and the hierarchical principle , 2017, IET Signal Process..

[14]  Tingli Su,et al.  Deep Hybrid Model Based on EMD with Classification by Frequency Characteristics for Long-Term Air Quality Prediction , 2020, Mathematics.

[15]  Durga Toshniwal,et al.  Empirical Mode Decomposition Based Deep Learning for Electricity Demand Forecasting , 2018, IEEE Access.

[16]  Chunping Chen,et al.  A New Iterative Least Squares Parameter Estimation Approach for Equation-error Autoregressive Systems , 2020, International Journal of Control, Automation and Systems.

[17]  Amr Badr,et al.  Forecasting of nonlinear time series using ANN , 2017 .

[18]  Feng Ding,et al.  Decomposition Least-Squares-Based Iterative Identification Algorithms for Multivariable Equation-Error Autoregressive Moving Average Systems , 2019, Mathematics.

[19]  Qian Sun,et al.  Spatio-Temporal Prediction for the Monitoring-Blind Area of Industrial Atmosphere Based on the Fusion Network , 2019, International journal of environmental research and public health.

[20]  Lei Yan,et al.  Hybrid Deep-Learning Framework Based on Gaussian Fusion of Multiple Spatiotemporal Networks for Walking Gait Phase Recognition , 2020, Complex..

[21]  F. Ding,et al.  Partially‐coupled least squares based iterative parameter estimation for multi‐variable output‐error‐like autoregressive moving average systems , 2019, IET Control Theory & Applications.

[22]  M. Gaglio,et al.  Ecosystem services approach for sustainable governance in a brackish water lagoon used for aquaculture , 2019, Journal of Environmental Planning and Management.

[23]  Jiabin Yu,et al.  A Hybrid Path Planning Method for an Unmanned Cruise Ship in Water Quality Sampling , 2019, IEEE Access.

[24]  Xiao Zhang,et al.  Hierarchical parameter and state estimation for bilinear systems , 2020, Int. J. Syst. Sci..

[25]  Yongming Pan,et al.  Research of Air Pollutant Concentration Forecasting Based on Deep Learning Algorithms , 2019, IOP Conference Series: Earth and Environmental Science.

[26]  Yan Ji,et al.  Hierarchical least squares parameter estimation algorithm for two-input Hammerstein finite impulse response systems , 2020, J. Frankl. Inst..

[27]  Jian-zhong Zhou,et al.  Upper and Lower Bound Interval Forecasting Methodology Based on Ideal Boundary and Multiple Linear Regression Models , 2019, Water Resources Management.

[28]  Feng Ding,et al.  Parameter estimation with scarce measurements , 2011, Autom..

[29]  Yegang Chen.  Prediction algorithm of PM2.5 mass concentration based on adaptive BP neural network , 2018, Computing.

[30]  Marcella Busilacchio,et al.  Recursive neural network model for analysis and forecast of PM10 and PM2.5 , 2017 .

[31]  Hashim Abu-gellban,et al.  Self-boosted Time-series Forecasting with Multi-task and Multi-view Learning , 2019, ArXiv.

[32]  Tao Yu,et al.  Parameter estimation for block‐oriented nonlinear systems using the key term separation , 2020, International Journal of Robust and Nonlinear Control.

[33]  Heng-Ming Tai,et al.  An Optimized Heterogeneous Structure LSTM Network for Electricity Price Forecasting , 2019, IEEE Access.

[34]  Ling Xu,et al.  The damping iterative parameter identification method for dynamical systems based on the sine signal measurement , 2016, Signal Process..

[35]  Xiangkui Wan,et al.  Two-stage Gradient-based Iterative Estimation Methods for Controlled Autoregressive Systems Using the Measurement Data , 2020, International Journal of Control, Automation and Systems.

[36]  Fang-Fang Li,et al.  Long term rolling prediction model for solar radiation combining empirical mode decomposition (EMD) and artificial neural network (ANN) techniques , 2018 .

[37]  Ling Xu,et al.  Highly computationally efficient state filter based on the delta operator , 2019, International Journal of Adaptive Control and Signal Processing.

[38]  Akihiko Noda.  A test of the adaptive market hypothesis using a time-varying AR model in Japan , 2012, 1207.1842.

[39]  Xiaochuan Pan,et al.  Attributable risk and economic cost of hospital admissions for mental disorders due to PM2.5 in Beijing. , 2020, The Science of the total environment.

[40]  Feng Ding,et al.  State estimation for bilinear systems through minimizing the covariance matrix of the state estimation errors , 2019, International Journal of Adaptive Control and Signal Processing.

[41]  Feng Ding,et al.  Novel data filtering based parameter identification for multiple-input multiple-output systems using the auxiliary model , 2016, Autom..

[42]  Zijian Liu,et al.  A deep learning based multitask model for network-wide traffic speed prediction , 2020, Neurocomputing.

[43]  Tingli Su,et al.  Integrated Predictor Based on Decomposition Mechanism for PM2.5 Long-Term Prediction , 2019, Applied Sciences.

[44]  Jie Li,et al.  Wavelet Decomposition and Convolutional LSTM Networks Based Improved Deep Learning Model for Solar Irradiance Forecasting , 2018, Applied Sciences.

[45]  Feng Ding,et al.  Modeling Nonlinear Processes Using the Radial Basis Function-Based State-Dependent Autoregressive Models , 2020, IEEE Signal Processing Letters.

[46]  Yu Guo,et al.  Structural parameter identification for 6 DOF industrial robots , 2017 .

[47]  Jianxin Wu,et al.  Minimal gated unit for recurrent neural networks , 2016, International Journal of Automation and Computing.

[48]  Wei Wei,et al.  Time-Delay System Control Based on an Integration of Active Disturbance Rejection and Modified Twice Optimal Control , 2019, IEEE Access.

[49]  Qinyao Liu,et al.  Recursive identification of bilinear time-delay systems through the redundant rule , 2020, J. Frankl. Inst..

[50]  T. Hayat,et al.  Hierarchical Parameter Estimation for the Frequency Response Based on the Dynamical Window Data , 2018, International Journal of Control, Automation and Systems.

[51]  Ling Xu,et al.  A Recursive Parameter Estimation Algorithm for Modeling Signals with Multi-frequencies , 2020, Circuits Syst. Signal Process..

[52]  Seyda Ertekin,et al.  Improving forecasting accuracy of time series data using a new ARIMA-ANN hybrid method and empirical mode decomposition , 2018, Neurocomputing.

[53]  Jiping Xu,et al.  A novel water quality mechanism modeling and eutrophication risk assessment method of lakes and reservoirs , 2019, Nonlinear Dynamics.

[54]  Xiaoming Xue,et al.  Short-Term Wind Speed Interval Prediction Based on Ensemble GRU Model , 2020, IEEE Transactions on Sustainable Energy.

[55]  Yuanxi Yang,et al.  Analysis of seasonal signals and long-term trends in the height time series of IGS sites in China , 2016, Science China Earth Sciences.

[56]  Yang Yang,et al.  Long-Term Span Traffic Prediction Model Based on STL Decomposition and LSTM , 2019, 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS).

[57]  Ling Xu,et al.  Separable multi‐innovation stochastic gradient estimation algorithm for the nonlinear dynamic responses of systems , 2020, International Journal of Adaptive Control and Signal Processing.

[58]  Ling Xu,et al.  Hierarchical Multi-Innovation Generalised Extended Stochastic Gradient Methods for Multivariable Equation-Error Autoregressive Moving Average Systems , 2020 .

[59]  Weide Li,et al.  Effective passenger flow forecasting using STL and ESN based on two improvement strategies , 2019, Neurocomputing.

[60]  Feng Ding,et al.  An efficient hierarchical identification method for general dual-rate sampled-data systems , 2014, Autom..

[61]  Feng Ding,et al.  Hierarchical Least Squares Identification for Linear SISO Systems With Dual-Rate Sampled-Data , 2011, IEEE Transactions on Automatic Control.

[62]  Ratnadip Adhikari,et al.  Time Series Forecasting Using Hybrid ARIMA and ANN Models Based on DWT Decomposition , 2015 .

[63]  Feng Ding,et al.  Partially Coupled Stochastic Gradient Identification Methods for Non-Uniformly Sampled Systems , 2010, IEEE Transactions on Automatic Control.

[64]  Ke Xu,et al.  High-order Hidden Markov Model for trend prediction in financial time series , 2019, Physica A: Statistical Mechanics and its Applications.

[65]  Graham W. Taylor,et al.  Forecasting air quality time series using deep learning , 2018, Journal of the Air & Waste Management Association.

[66]  Ahmed Alsaedi,et al.  Gradient estimation algorithms for the parameter identification of bilinear systems using the auxiliary model , 2020, J. Comput. Appl. Math..