N-BEATS: Neural basis expansion analysis for interpretable time series forecasting

We focus on solving the univariate times series point forecasting problem using deep learning. We propose a deep neural architecture based on backward and forward residual links and a very deep stack of fully-connected layers. The architecture has a number of desirable properties, being interpretable, applicable without modification to a wide array of target domains, and fast to train. We test the proposed architecture on several well-known datasets, including M3, M4 and TOURISM competition datasets containing time series from diverse domains. We demonstrate state-of-the-art performance for two configurations of N-BEATS for all the datasets, improving forecast accuracy by 11% over a statistical benchmark and by 3% over last year's winner of the M4 competition, a domain-adjusted hand-crafted hybrid between neural network and statistical time series models. The first configuration of our model does not employ any time-series-specific components and its performance on heterogeneous datasets strongly suggests that, contrarily to received wisdom, deep learning primitives such as residual blocks are by themselves sufficient to solve a wide range of forecasting problems. Finally, we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy.

[1]  Peter R. Winters,et al.  Forecasting Sales by Exponentially Weighted Moving Averages , 1960 .

[2]  Robert L. Winkler,et al.  The accuracy of extrapolation (time series) methods: Results of a forecasting competition , 1982 .

[3]  Irma J. Terpenning,et al.  STL : A Seasonal-Trend Decomposition Procedure Based on Loess , 1990 .

[4]  Yoshua Bengio,et al.  Learning a synaptic learning rule , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[5]  K. Nikolopoulos,et al.  The theta model: a decomposition approach to forecasting , 2000 .

[6]  Spyros Makridakis,et al.  The M3-Competition: results, conclusions and implications , 2000 .

[7]  Kenneth B. Kahn How to Measure the Impact of a Forecast Error on an Enterprise , 2003 .

[8]  C. Holt Author's retrospective on ‘Forecasting seasonals and trends by exponentially weighted moving averages’ , 2004 .

[9]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[10]  Aris A. Syntetos,et al.  On the categorization of demand patterns , 2005, J. Oper. Res. Soc..

[11]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[12]  Rob J Hyndman,et al.  Automatic Time Series Forecasting: The forecast Package for R , 2008 .

[13]  Konstantinos Nikolopoulos,et al.  The Tourism Forecasting Competition , 2011 .

[14]  C. L. Jain,et al.  Answers to Your Forecasting Questions , 2010 .

[15]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[16]  P. Brierley Winning methods for forecasting seasonal tourism time series , 2011 .

[17]  Rob J Hyndman,et al.  The value of feedback in forecasting competitions , 2011 .

[18]  Lee C. Baker,et al.  Winning methods for forecasting tourism time series , 2011 .

[19]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Rob J Hyndman,et al.  Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation , 2016 .

[22]  Inderjit S. Dhillon,et al.  Temporal Regularized Matrix Factorization for High-dimensional Time Series Prediction , 2016, NIPS.

[23]  A. Koehler,et al.  Models for optimising the theta method and their relationship to state space models , 2016 .

[24]  Jungwon Lee,et al.  Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition , 2017, INTERSPEECH.

[25]  Thomas S. Huang,et al.  Dilated Recurrent Neural Networks , 2017, NIPS.

[26]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[27]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Garrison W. Cottrell,et al.  A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction , 2017, IJCAI.

[29]  Tehseen Zia,et al.  Residual Recurrent Highway Networks for Learning Deep Sequence Prediction Models , 2018, Journal of Grid Computing.

[30]  Evangelos Spiliotis,et al.  Statistical and Machine Learning forecasting methods: Concerns and ways forward , 2018, PloS one.

[31]  Matthias W. Seeger,et al.  Deep State Space Models for Time Series Forecasting , 2018, NeurIPS.

[32]  Evangelos Spiliotis,et al.  The M4 Competition: Results, findings, conclusion and way forward , 2018, International Journal of Forecasting.

[33]  Tim Januschowski,et al.  Deep Factors for Forecasting , 2019, ICML.

[34]  Jean-François Toubeau,et al.  Deep Learning-Based Multivariate Probabilistic Forecasting for Short-Term Scheduling in Power Markets , 2019, IEEE Transactions on Power Systems.

[35]  Evangelos Spiliotis,et al.  Forecasting with a hybrid method utilizing data smoothing, a variation of the Theta method and shrinkage of seasonal factors , 2018, International Journal of Production Economics.

[36]  Slawek Smyl,et al.  A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting , 2020, International Journal of Forecasting.

[37]  Rob J. Hyndman,et al.  FFORMA: Feature-based forecast model averaging , 2020, International Journal of Forecasting.