A Survey of Estimation Methods for Sparse High-dimensional Time Series Models

High-dimensional time series datasets are becoming increasingly common in many areas of biological and social sciences. Some important applications include gene regulatory network reconstruction using time course gene expression data, brain connectivity analysis from neuroimaging data, structural analysis of a large panel of macroeconomic indicators, and studying linkages among financial firms for more robust financial regulation. These applications have led to renewed interest in developing principled statistical methods and theory for estimating large time series models given only a relatively small number of temporally dependent samples. Sparse modeling approaches have gained popularity over the last two decades in statistics and machine learning for their interpretability and predictive accuracy. Although there is a rich literature on several sparsity inducing methods when samples are independent, research on the statistical properties of these methods for estimating time series models is still in progress. We survey some recent advances in this area, focusing on empirically successful lasso based estimation methods for two canonical multivariate time series models stochastic regression and vector autoregression. We discuss key technical challenges arising in high-dimensional time series analysis and outline several interesting research directions.

[1]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[2]  Jianqing Fan,et al.  Regularity Properties of High-dimensional Covariate Matrices ∗ , 2013 .

[3]  J. Davidson Stochastic Limit Theory: An Introduction for Econometricians , 1994 .

[4]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .

[5]  I. Simon,et al.  Studying and modelling dynamic biological processes using time-series gene expression data , 2012, Nature Reviews Genetics.

[6]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[7]  A. Lo,et al.  A Survey of Systemic Risk Analytics , 2012 .

[8]  M. Rudelson,et al.  Hanson-Wright inequality and sub-gaussian concentration , 2013 .

[9]  Ambuj Tewari,et al.  Regularized Estimation in High Dimensional Time Series under Mixing Conditions , 2016, ArXiv.

[10]  Ellis W. Tallman,et al.  Improving forecasts of the federal funds rate in a policy model , 1999 .

[11]  Robert B. Litterman A Statistical Approach to Economic Forecasting , 1986 .

[12]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[13]  David S. Matteson,et al.  Interpretable vector autoregressions with exogenous time series , 2017, 1711.03623.

[14]  G. Koop Forecasting with Medium and Large Bayesian VARs , 2013 .

[15]  Martin J. Wainwright,et al.  Estimation of (near) low-rank matrices with noise and high-dimensional scaling , 2009, ICML.

[16]  Fang Han,et al.  Transition Matrix Estimation in High Dimensional Time Series , 2013, ICML.

[17]  William B. Nicholson,et al.  VARX-L: Structured Regularization for Large Vector Autoregressions with Exogenous Variables , 2015, 1508.07497.

[18]  Y. Wu,et al.  Performance bounds for parameter estimates of high-dimensional linear models with correlated errors , 2016 .

[19]  Sumanta Basu,et al.  Modeling and Estimation of High-dimensional Vector Autoregressions. , 2014 .

[20]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[21]  Garvesh Raskutti,et al.  Network Estimation From Point Process Data , 2018, IEEE Transactions on Information Theory.

[22]  Rebecca Willett,et al.  Inference of High-dimensional Autoregressive Generalized Linear Models , 2016, ArXiv.

[23]  Po-Ling Loh,et al.  High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity , 2011, NIPS.

[24]  P. Bickel,et al.  Large Vector Auto Regressions , 2011, 1106.3915.

[25]  Jianqing Fan,et al.  Risks of Large Portfolios , 2013, Journal of econometrics.

[26]  Xiaohui Chen,et al.  Regularized Estimation of Linear Functionals of Precision Matrices for High-Dimensional Time Series , 2015, IEEE Transactions on Signal Processing.

[27]  Danielle S Bassett,et al.  Brain graphs: graphical models of the human brain connectome. , 2011, Annual review of clinical psychology.

[28]  Christophe Croux,et al.  Identifying demand effects in a large network of product categories , 2015, 1506.01589.

[29]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[30]  G. Michailidis,et al.  Regularized estimation in sparse high-dimensional time series models , 2013, 1311.4175.

[31]  Nan-Jung Hsu,et al.  Subset selection for vector autoregressive processes using Lasso , 2008, Comput. Stat. Data Anal..

[32]  Robert B. Litterman Techniques of forecasting using vector autoregressions , 1979 .

[33]  S. Geer,et al.  The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso) , 2011 .

[34]  Martin J. Wainwright,et al.  Restricted Eigenvalue Properties for Correlated Gaussian Designs , 2010, J. Mach. Learn. Res..

[35]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[36]  David S. Matteson,et al.  Sparse Identification and Estimation of Large-Scale Vector AutoRegressive Moving Averages , 2017, Journal of the American Statistical Association.

[37]  Kam Chung Wong,et al.  Lasso guarantees for $\beta$-mixing heavy-tailed time series , 2017, 1708.01505.

[38]  Richard A. Davis,et al.  Sparse Vector Autoregressive Modeling , 2012, 1207.0520.

[39]  Cun-Hui Zhang,et al.  Scaled sparse linear regression , 2011, 1104.4595.

[40]  Deborah Gefang,et al.  Bayesian doubly adaptive elastic-net Lasso for VAR shrinkage , 2014 .

[41]  R. C. Bradley Basic properties of strong mixing conditions. A survey and some open questions , 2005, math/0511078.

[42]  Arindam Banerjee,et al.  Estimating Structured Vector Autoregressive Models , 2016, ICML.

[43]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[44]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[45]  C. Sims MACROECONOMICS AND REALITY , 1977 .

[46]  Michael Schweinberger,et al.  High-Dimensional Multivariate Time Series With Additional Structure , 2015, 1510.02159.

[47]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[48]  H. Zou The Adaptive Lasso and Its Oracle Properties , 2006 .

[49]  K. R. Kadiyala,et al.  Numerical Methods for Estimation and Inference in Bayesian VAR-models , 1997 .

[50]  Jiahan Li,et al.  Forecasting Macroeconomic Time Series: LASSO-Based Approaches and Their Forecast Combinations with Dynamic Factor Models , 2014 .

[51]  M. Rosenblatt A CENTRAL LIMIT THEOREM AND A STRONG MIXING CONDITION. , 1956, Proceedings of the National Academy of Sciences of the United States of America.

[52]  C. De Mol,et al.  Forecasting Using a Large Number of Predictors: Is Bayesian Regression a Valid Alternative to Principal Components? , 2006, SSRN Electronic Journal.

[53]  D. B. Preston Spectral Analysis and Time Series , 1983 .

[54]  Bin Yu RATES OF CONVERGENCE FOR EMPIRICAL PROCESSES OF STATIONARY MIXING SEQUENCES , 1994 .

[55]  William B. Nicholson,et al.  BigVAR: Tools for Modeling Sparse High-Dimensional Multivariate Time Series , 2017, 1702.07094.

[56]  Dominique M. Hanssens,et al.  Do Promotions Benefit Manufacturers, Retailers, or Both? , 2002, Manag. Sci..

[57]  Eric T. Shea-Brown,et al.  The Multivariate Hawkes Process in High Dimensions: Beyond Mutual Excitation , 2017, 1707.04928.

[58]  Cun-Hui Zhang Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[59]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[60]  W. Wu,et al.  Covariance and precision matrix estimation for high-dimensional time series , 2013, 1401.0993.

[61]  Fang Han,et al.  Robust Estimation of Transition Matrices in High Dimensional Heavy-tailed Vector Autoregressive Processes , 2015, ICML.

[62]  W. Wu,et al.  Nonlinear system theory: another look at dependence. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[63]  Eduardo F. Mendes,et al.  ℓ1-regularization of high-dimensional time-series models with non-Gaussian and heteroskedastic errors , 2016 .

[64]  D. Giannone,et al.  Large Bayesian vector auto regressions , 2010 .

[65]  W. Wu,et al.  Asymptotic theory for stationary processes , 2011 .

[66]  A. Belloni,et al.  Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming , 2010, 1009.5689.