Cluster-and-Conquer: A Framework For Time-Series Forecasting

We propose a three-stage framework for forecasting high-dimensional time-series data. Our method first estimates parameters for each univariate time series. Next, we use these parameters to cluster the time series. These clusters can be viewed as multivariate time series, for which we then compute parameters. The forecasted values of a single time series can depend on the history of other time series in the same cluster, accounting for intra-cluster similarity while minimizing potential noise in predictions by ignoring intercluster effects. Our framework—which we refer to as “cluster-and-conquer”—is highly general, allowing for any time-series forecasting and clustering method to be used in each step. It is computationally efficient and embarrassingly parallel.We motivate our framework with a theoretical analysis in an idealized mixed linear regression setting, where we provide guarantees on the quality of the estimates. We accompany these guarantees with experimental results that demonstrate the advantages of our framework: when instantiated with simple linear autoregressive models, we are able to achieve state-of-the-art results on several benchmark datasets, sometimes outperforming deep-learning-based approaches.

[1]  Qi Zhang,et al.  Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting , 2020, NeurIPS.

[2]  Guokun Lai,et al.  Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks , 2017, SIGIR.

[3]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[4]  Michael W. Mahoney,et al.  Noisy Recurrent Neural Networks , 2021, NeurIPS.

[5]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[6]  Santosh S. Vempala,et al.  Spectral Algorithms , 2009, Found. Trends Theor. Comput. Sci..

[7]  Anderson Y. Zhang,et al.  Optimality of Spectral Clustering for Gaussian Mixture Model , 2019, ArXiv.

[8]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[9]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[10]  Christos Faloutsos,et al.  FUNNEL: automatic mining of spatially coevolving epidemics , 2014, KDD.

[11]  Syama Sundar Rangapuram,et al.  Neural forecasting: Introduction and literature overview , 2020, ArXiv.

[12]  A. Zeevi,et al.  NONASYMPTOTIC BOUNDS FOR AUTOREGRESSIVE TIME SERIES MODELING , 2001 .

[13]  Shaojun Guo,et al.  High Dimensional and Banded Vector Autoregressions , 2015, 1502.07831.

[14]  Jing Jiang,et al.  Graph WaveNet for Deep Spatial-Temporal Graph Modeling , 2019, IJCAI.

[15]  Omri Azencot,et al.  Forecasting Sequential Data using Consistent Koopman Autoencoders , 2020, ICML.

[16]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[17]  Hsiang-Fu Yu,et al.  Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting , 2019, NeurIPS.

[18]  Eamonn J. Keogh,et al.  The UCR time series archive , 2018, IEEE/CAA Journal of Automatica Sinica.

[19]  Inderjit S. Dhillon,et al.  Temporal Regularized Matrix Factorization for High-dimensional Time Series Prediction , 2016, NIPS.

[20]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[21]  Pravin Varaiya,et al.  Freeway Performance Measurement System: Mining Loop Detector Data , 2001 .

[22]  Lina Yao,et al.  Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting , 2020, NeurIPS.

[23]  Chellu Chandra Sekhar,et al.  Bayesian Mixture of AR Models for Time Series Clustering , 2009, 2009 Seventh International Conference on Advances in Pattern Recognition.

[24]  Cosma Rohilla Shalizi,et al.  Nonparametric Risk Bounds for Time-Series Forecasting , 2012, J. Mach. Learn. Res..

[25]  Qiang Fu,et al.  YADING: Fast Clustering of Large-Scale Time Series Data , 2015, Proc. VLDB Endow..

[26]  Christos Faloutsos,et al.  Forecasting Big Time Series: Theory and Practice , 2019, KDD.

[27]  Sander Bohte,et al.  Conditional Time Series Forecasting with Convolutional Neural Networks , 2017, 1703.04691.

[28]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[29]  Vandana,et al.  Survey of Nearest Neighbor Techniques , 2010, ArXiv.

[30]  Nicolas Chapados,et al.  N-BEATS: Neural basis expansion analysis for interpretable time series forecasting , 2019, ICLR.

[31]  George Athanasopoulos,et al.  Forecasting: principles and practice , 2013 .

[32]  Ying Wah Teh,et al.  Time-series clustering - A decade review , 2015, Inf. Syst..

[33]  Matthias W. Seeger,et al.  Deep State Space Models for Time Series Forecasting , 2018, NeurIPS.

[34]  Konstantinos Kalpakis,et al.  Distance measures for effective clustering of ARIMA time-series , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[35]  Dennis Shasha,et al.  StatStream: Statistical Monitoring of Thousands of Data Streams in Real Time , 2002, VLDB.

[36]  Omri Azencot,et al.  Lipschitz Recurrent Neural Networks , 2020, ICLR.

[37]  Michael W. Mahoney,et al.  Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction , 2019, ArXiv.

[38]  Mehryar Mohri,et al.  Learning Theory and Algorithms for Forecasting Non-stationary Time Series , 2015, NIPS.

[39]  Christoph Bergmeir,et al.  Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach , 2017, Expert Syst. Appl..

[40]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.