Empirical Risk Minimization for Time Series: Nonparametric Performance Bounds for Prediction

Empirical risk minimization is a standard principle for choosing algorithms in learning theory. In this paper we study the properties of empirical risk minimization for time series. The analysis is carried out in a general framework that covers different types of forecasting applications encountered in the literature. We are concerned with 1-step-ahead prediction of a univariate time series generated by a parameter-driven process. A class of recursive algorithms is available to forecast the time series. The algorithms are recursive in the sense that the forecast produced in a given period is a function of the lagged values of the forecast and of the time series. The relationship between the generating mechanism of the time series and the class of algorithms is unspecified. Our main result establishes that the algorithm chosen by empirical risk minimization achieves asymptotically the optimal predictive performance that is attainable within the class of algorithms.

[1]  Andrew Harvey,et al.  Beta-t-(E)GARCH , 2008 .

[2]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[3]  Pentti Saikkonen,et al.  ERGODICITY, MIXING, AND EXISTENCE OF MOMENTS OF A CLASS OF MARKOV MODELS WITH APPLICATIONS TO GARCH AND ACD MODELS , 2008, Econometric Theory.

[4]  Irène Gijbels,et al.  Understanding exponential smoothing via kernel regression , 1999 .

[5]  Michael McAleer,et al.  ASYMPTOTIC THEORY FOR A VECTOR ARMA-GARCH MODEL , 2003, Econometric Theory.

[6]  Adrian Pagan,et al.  Alternative Models for Conditional Stock Volatility , 1989 .

[7]  On Stationarity and Ergodicity of the Bilinear Model with Applications to GARCH Models , 2005 .

[8]  C. Morris.  Natural Exponential Families with Quadratic Variance Functions , 1982 .

[9]  M. Tanner,et al.  RISK MINIMIZATION FOR TIME SERIES BINARY CHOICE WITH VARIABLE SELECTION , 2010, Econometric Theory.

[10]  N. Meddahi,et al.  ARMA representation of integrated and realized variances , 2003 .

[11]  Nalini Ravishanker,et al.  Count Time Series: A Methodological Review , 2021 .

[12]  H. Tong.  Non-linear time series. A dynamical system approach , 1990 .

[13]  Drew D. Creal,et al.  Generalized autoregressive score models with applications ∗ , 2010 .

[14]  Andrew J. Patton.  Comparing Possibly Misspecified Forecasts , 2020, Journal of Business & Economic Statistics.

[15]  Bruce E. Hansen,et al.  Asymptotic Theory for the Garch(1,1) Quasi-Maximum Likelihood Estimator , 1994, Econometric Theory.

[16]  P. Bougerol,et al.  Stationarity of Garch processes and of some nonnegative time series , 1992 .

[17]  Anders Rahbek,et al.  Modeling corporate defaults: Poisson autoregressions with exogenous covariates (PARX) , 2016 .

[18]  Zudi Lu,et al.  L1 geometric ergodicity of a multivariate nonlinear AR model with an ARCH term , 2001 .

[19]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001 .

[20]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[21]  H. White,et al.  A Unified Theory of Estimation and Inference for Nonlinear Dynamic Models , 1988 .

[22]  D. Tjøstheim,et al.  Nonparametric Estimation and Identification of Nonlinear ARCH Time Series Strong Convergence and Asymptotic Normality: Strong Convergence and Asymptotic Normality , 1995, Econometric Theory.

[23]  Andrew J. Patton.  Volatility Forecast Comparison Using Imperfect Volatility Proxies , 2006 .

[24]  P. Hansen,et al.  Consistent Ranking of Volatility Models , 2006 .

[25]  Wolfgang Karl Härdle,et al.  Local polynomial estimators of the volatility function in nonparametric autoregression , 1997 .

[26]  R. Lumsdaine,et al.  Consistency and Asymptotic Normality of the Quasi-maximum Likelihood Estimator in IGARCH(1,1) and Covariance Stationary GARCH(1,1) Models , 1996 .

[27]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[28]  J. Zakoian,et al.  GARCH Models: Structure, Statistical Inference and Financial Applications , 2010 .

[29]  J. Zakoian,et al.  Maximum likelihood estimation of pure GARCH and ARMA-GARCH processes , 2004 .

[30]  Anders Rahbek,et al.  ASYMPTOTICS OF THE QMLE FOR A CLASS OF ARCH(q) MODELS , 2005, Econometric Theory.

[31]  Eckhard Liebscher,et al.  Strong convergence of sums of α-mixing random variables with applications to density estimation , 1996 .

[32]  Dean P. Foster,et al.  Filtering and Forecasting with Misspecified Arch Models Ii: Making the Right Forecast with the Wrong Model , 1992 .

[33]  N. Meddahi,et al.  A theoretical comparison between integrated and realized volatility , 2002 .

[34]  E. Liebscher.  Towards a Unified Approach for Proving Geometric Ergodicity and Mixing Properties of Nonlinear Autoregressive Processes , 2005 .

[35]  Enno Mammen,et al.  Estimating Semiparametric Arch (∞) Models by Kernel Smoothing Methods , 2003 .

[36]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[37]  B. M. Pötscher,et al.  Dynamic Nonlinear Econometric Models: Asymptotic Theory , 1997 .

[38]  A. Harvey.  Dynamic Models for Volatility and Heavy Tails: With Applications to Financial and Economic Time Series , 2013 .

[39]  Francis X. Diebold,et al.  Modeling and Forecasting Realized Volatility , 2001 .

[40]  Xiaohong Chen,et al.  MIXING AND MOMENT PROPERTIES OF VARIOUS GARCH AND STOCHASTIC VOLATILITY MODELS , 2002, Econometric Theory.

[41]  Jeffrey R. Russell,et al.  Autoregressive Conditional Duration: A New Model for Irregularly Spaced Transaction Data , 1998 .

[42]  Jeroen V.K. Rombouts,et al.  Série Scientifique Scientific Series on Loss Functions and Ranking Forecasting Performances of Multivariate Volatility Models on Loss Functions and Ranking Forecasting Performances of Multivariate Volatility Models , 2022 .

[43]  Christian Brownlees,et al.  Performance of Empirical Risk Minimization for Linear Regression with Dependent Data , 2021, SSRN Electronic Journal.

[44]  R. Engle,et al.  A Multiple Indicators Model for Volatility Using Intra-Daily Data , 2003 .

[45]  Thomas Mikosch,et al.  Quasi-maximum-likelihood estimation in conditionally heteroscedastic time series: A stochastic recurrence equations approach , 2006 .

[46]  Daniel B. Nelson,et al.  Filtering and Forecasting with Misspecified Arch Models Ii: Making the Right Forecast with the Wrong Model , 1992 .

[47]  J. Rosenthal,et al.  General state space Markov chains and MCMC algorithms , 2004, math/0404033.

[48]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[49]  Pentti Saikkonen,et al.  Non-Linear GARCH Models for Highly Persistent Volatility , 2005 .

[50]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[51]  Andrew J. Patton,et al.  Comparing Possibly Misspeci ed Forecasts , 2014 .

[52]  Christian Francq,et al.  MIXING PROPERTIES OF A GENERAL CLASS OF GARCH(1,1) MODELS WITHOUT MOMENT ASSUMPTIONS ON THE OBSERVED PROCESS , 2006, Econometric Theory.

[53]  J. Davidson.  Stochastic Limit Theory , 1994 .

[54]  Shahar Mendelson,et al.  Learning without Concentration , 2014, COLT.