Online ARIMA Algorithms for Time Series Prediction

Autoregressive integrated moving average (ARIMA) is one of the most popular linear models for time series forecasting due to its nice statistical properties and great flexibility. However, its parameters are estimated in a batch manner and its noise terms are often assumed to be strictly bounded, which restricts its applications and makes it inefficient for handling large-scale real data. In this paper, we propose online learning algorithms for estimating ARIMA models under relaxed assumptions on the noise terms, which is suitable to a wider range of applications and enjoys high computational efficiency. The idea of our ARIMA method is to reformulate the ARIMA model into a task of full information online optimization (without random noise terms). As a consequence, we can online estimation of the parameters in an efficient and scalable way. Furthermore, we analyze regret bounds of the proposed algorithms, which guarantee that our online ARIMA model is provably as good as the best ARIMA model in hindsight. Finally, our encouraging experimental results further validate the effectiveness and robustness of our method.

[1]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[2]  Bows Abraham,et al.  Analysis of Economic Time Series: A Synthesis , 1979 .

[3]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[4]  Steven C. H. Hoi,et al.  LIBOL: a library for online learning algorithms , 2014, J. Mach. Learn. Res..

[5]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[6]  Feng Ding,et al.  Performance analysis of estimation algorithms of nonstationary ARMA processes , 2006, IEEE Transactions on Signal Processing.

[7]  James D. Hamilton Time Series Analysis , 1994 .

[8]  Steven C. H. Hoi,et al.  Online Portfolio Selection , 2015 .

[9]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[10]  R. Engle Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[11]  Ken-Shin Lii,et al.  Identification and estimation of non-Gaussian ARMA processes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[12]  Shie Mannor,et al.  Online Learning for Time Series Prediction , 2013, COLT.

[13]  G. C. Tiao,et al.  Consistent Estimates of Autoregressive Parameters and Extended Sample Autocorrelation Function for Stationary and Nonstationary ARMA Models , 1984 .

[14]  Ruey S. Tsay,et al.  Analysis of Financial Time Series: Tsay/Analysis of Financial Time Series , 2005 .

[15]  C. Granger,et al.  Forecasting Economic Time Series. , 1988 .

[16]  Shyh-Jier Huang,et al.  Short-term load forecasting via ARMA model identification including non-Gaussian process considerations , 2003 .

[17]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[18]  José Luis Rojo-Álvarez,et al.  Support vector method for robust ARMA system identification , 2004, IEEE Transactions on Signal Processing.

[19]  M. Tiku,et al.  Time Series Models in Non‐Normal Situations: Symmetric Innovations , 2000 .

[20]  Sébastien Bubeck,et al.  Introduction to Online Optimization , 2011 .

[21]  K. Lii Model Identification and Estimation of NonGaussian ARMA Processes. , 1982 .

[22]  A. Nejat Ince,et al.  Digital Speech Processing , 1992 .

[23]  A. El-Shaarawi,et al.  ARMA Models with Double-exponentially Distributed Noise , 1989 .

[25]  Jing Hu,et al.  Denoising Nonlinear Time Series by Adaptive Filtering and Wavelet Shrinkage: A Comparison , 2010, IEEE Signal Processing Letters.

[26]  David M. Grether,et al.  Analysis of Economic Time Series. A Synthesis. , 1980 .

[27]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.