An Ensemble Model of Arima and Ann with Restricted Boltzmann Machine Based on Decomposition of Discrete Wavelet Transform for Time Series Forecasting

Time series forecasting research area mainly focuses on developing effective forecasting models to improve prediction accuracy. An ensemble model composed of autoregressive integrated moving average (ARIMA), artificial neural network (ANN), restricted Boltzmann machines (RBM), and discrete wavelet transform (DWT) is presented in this paper. In the proposed model, DWT first decomposes time series into approximation and detail. Then Khashei and Bijari’s model, which is an ensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their both linear and nonlinear components and fit the relationship between the components as a function instead of additive relationship. Furthermore, RBM is used to perform pre-training for generating initial weights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detail are combined to obtain final forecasting. The forecasting capability of the proposed model is tested with three well-known time series: sunspot, Canadian lynx, exchange rate time series. The prediction performance is compared to the other six forecasting models. The results indicate that the proposed model gives the best performance in all three data sets and all three measures (i.e. MSE, MAE and MAPE).

[1]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[2]  A. M. Walker,et al.  A Survey of statistical work on the Mackenzie River series of annual Canadian lynx trappings for the , 1977 .

[3]  Wei Gao,et al.  Learning Granger causality graphs for multivariate nonlinear time series , 2009 .

[4]  Xijin J. Tang,et al.  Ensemble of multiple kNN classifiers for societal risk classification , 2017, Journal of Systems Science and Systems Engineering.

[5]  Mohsen Pourahmadi,et al.  Nonparametric and non‐linear models and data mining in time series: a case‐study on the Canadian lynx data , 2008 .

[6]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[7]  Chandranath Chatterjee,et al.  Development of an accurate and reliable hourly flood forecasting model using wavelet–bootstrap–ANN (WBANN) hybrid approach , 2010 .

[8]  Gerhard Thury,et al.  Industrial production in Germany and Austria: A case study in structural time series modelling , 2003 .

[9]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  O. Kisi,et al.  Wavelet and neuro-fuzzy conjunction model for precipitation forecasting , 2007 .

[11]  Nanua Singh,et al.  Forecasting automobile warranty performance in presence of ‘maturing data’ phenomena using multilayer perceptron neural network , 2005 .

[12]  Jessica Fridrich,et al.  Modeling and Extending the Ensemble Classifier for Steganalysis of Digital Images Using Hypothesis Testing Theory , 2015, IEEE Transactions on Information Forensics and Security.

[13]  Ratnadip Adhikari,et al.  Time Series Forecasting Using Hybrid ARIMA and ANN Models Based on DWT Decomposition , 2015 .

[14]  W. Li,et al.  On a mixture autoregressive model , 2000 .

[15]  Bhekisipho Twala,et al.  Combining classifiers for credit risk prediction , 2009 .

[16]  Kunikazu Kobayashi,et al.  Time series forecasting using a deep belief network with restricted Boltzmann machines , 2014, Neurocomputing.

[17]  J. Adamowski,et al.  A wavelet neural network conjunction model for groundwater level forecasting , 2011 .

[18]  Kenneth S. Rogoff,et al.  Exchange rate models of the seventies. Do they fit out of sample , 1983 .

[19]  Vahid Nourani,et al.  A Multivariate ANN-Wavelet Approach for Rainfall–Runoff Modeling , 2009 .

[20]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[21]  Xiaoyi Jiang,et al.  Dynamic classifier ensemble model for customer classification with imbalanced class distribution , 2012, Expert Syst. Appl..

[22]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[23]  Van-Nam Huynh,et al.  A New Hybrid Linear-Nonlinear Model Based on Decomposition of Discrete Wavelet Transform for Time Series Forecasting , 2017 .

[24]  Rob J Hyndman,et al.  25 years of time series forecasting , 2006 .

[25]  Qing Liu,et al.  A quantum multi-agent based neural network model for failure prediction , 2016 .

[26]  Mi Zhou,et al.  Calendar effects in monthly time series models , 2005 .

[27]  N. J. Fliege,et al.  Multirate Digital Signal Processing: Multirate Systems, Filter Banks, Wavelets , 1994 .

[28]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[29]  Yong Peng,et al.  The Research of Monthly Discharge Predictor-corrector Model Based on Wavelet Decomposition , 2008 .

[30]  Hecht-Nielsen.  Theory of the backpropagation neural network , 1989 .

[31]  Judith E. Dayhoff,et al.  Neural Network Architectures: An Introduction , 1989 .

[32]  Depeng Zuo,et al.  Improving prediction accuracy of river discharge time series using a Wavelet-NAR artificial neural network , 2012 .

[33]  Ponnuthurai Nagaratnam Suganthan,et al.  Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting , 2017, Appl. Soft Comput..

[34]  Abdollah Kavousi-Fard,et al.  A hybrid method based on wavelet, ANN and ARIMA model for short-term load forecasting , 2014, J. Exp. Theor. Artif. Intell..

[35]  Z. Tan,et al.  Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models , 2010 .

[36]  A.J. Conejo,et al.  Day-ahead electricity price forecasting using the wavelet transform and ARIMA models , 2005, IEEE Transactions on Power Systems.

[37]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[38]  Mehdi Khashei,et al.  A novel hybridization of artificial neural networks and ARIMA models for time series forecasting , 2011, Appl. Soft Comput..

[39]  Jan Adamowski,et al.  Development of a coupled wavelet transform and neural network method for flow forecasting of non-perennial rivers in semi-arid watersheds. , 2010 .