The Model-Free Prediction Principle

The problem of prediction is revisited with a view towards going beyond the typical nonparametric setting and reaching a fully model-free environment for predictive inference, i.e., point predictors and predictive intervals. A basic principle of Model-free prediction is laid out based on the notion of transforming a given setup into one that is easier to work with, namely i.i.d. or Gaussian. The Model-Free Prediction Principle is defined, and its applicability to different settings is outlined.

[1]  H. Markowitz,et al.  The Random Character of Stock Market Prices. , 1965 .

[2]  Seymour Geisser,et al.  8. Predictive Inference: An Introduction , 1995 .

[3]  Siem Jan Koopman,et al.  Forecasting Daily Variability of the S&P 100 Stock Index Using Historical, Realised and Implied Volatility Measurements , 2005 .

[4]  James D. Hamilton.  Time Series Analysis , 1994 .

[5]  D. Politis,et al.  On the range of validity of the autoregressive sieve bootstrap , 2012, 1201.6211.

[6]  B. Efron.  Estimation and Accuracy After Model Selection , 2014, Journal of the American Statistical Association.

[7]  D. Politis,et al.  Local block bootstrap inference for trending time series , 2013 .

[8]  Yongmiao Hong,et al.  Hypothesis Testing in Time Series via the Empirical Characteristic Function: A Generalized Spectral Density Approach , 1999 .

[9]  M. Rosenblatt.  Remarks on a Multivariate Transformation , 1952 .

[10]  P. Bertail,et al.  Regenerative block bootstrap for Markov chains , 2006 .

[11]  Piotr Kokoszka,et al.  SEQUENTIAL CHANGE-POINT DETECTION IN GARCH(p,q) MODELS , 2004, Econometric Theory.

[12]  Max Kuhn,et al.  Applied Predictive Modeling , 2013 .

[13]  N. Meddahi,et al.  An Eigenfunction Approach for Volatility Modeling , 2001 .

[14]  A. Ullah,et al.  Nonparametric Econometrics , 1999 .

[15]  P. Hansen,et al.  Consistent Ranking of Volatility Models , 2006 .

[16]  Guido Masarotto,et al.  Bootstrap prediction intervals for autoregressions , 1990 .

[17]  T. Tony Cai,et al.  Effect of mean on variance function estimation in nonparametric regression , 2006 .

[18]  E. Nadaraya.  On Estimating Regression , 1964 .

[19]  Harrison H. Zhou,et al.  Optimal rates of convergence for estimating Toeplitz covariance matrices , 2013 .

[20]  D. Politis,et al.  Financial Time Series and Volatility Prediction using NoVaS Transformations , 2008 .

[21]  C. Granger,et al.  Nonstationarities in Stock Returns , 2005, Review of Economics and Statistics.

[22]  M. Pourahmadi.  Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation , 1999 .

[23]  D. Politis,et al.  The local bootstrap for Markov processes , 2002 .

[24]  James Stephen Marron,et al.  BOOTSTRAP SIMULTANEOUS ERROR BARS FOR NONPARAMETRIC REGRESSION , 1991 .

[25]  Jeffrey D. Hart,et al.  Nonparametric Smoothing and Lack-Of-Fit Tests , 1997 .

[26]  M. Akritas,et al.  Non‐parametric Estimation of the Residual Distribution , 2001 .

[27]  E. Ghysels,et al.  Série Scientifique Scientific Series Predicting Volatility: Getting the Most out of Return Data Sampled at Different Frequencies , 2022 .

[28]  T. Bollerslev,et al.  ANSWERING THE SKEPTICS: YES, STANDARD VOLATILITY MODELS DO PROVIDE ACCURATE FORECASTS* , 1998 .

[29]  R. Tibshirani.  Estimating Transformations for Regression via Additivity and Variance Stabilization , 1988 .

[30]  A. Dasgupta.  Asymptotic Theory of Statistics and Probability , 2008 .

[31]  Richard A. Davis,et al.  Time Series: Theory and Methods (2nd ed.). , 1992 .

[32]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[33]  R. Dahlhaus.  Fitting time series models to nonstationary processes , 1997 .

[34]  R. Koenker.  Quantile Regression: Name Index , 2005 .

[35]  ByoungSeon Choi,et al.  Arma Model Identification , 1992 .

[36]  D. Ruppert,et al.  Transformation and Weighting in Regression , 1988 .

[37]  David Ruppert,et al.  Bias reduction in kernel density estimation by smoothed empirical transformations , 1994 .

[38]  R. Dahlhaus,et al.  Statistical inference for time-varying ARCH processes , 2006, math/0607799.

[39]  Wenceslao González-Manteiga,et al.  Saving computer time in constructing consistent bootstrap prediction intervals for autoregressive processes , 1997 .

[40]  M. Pourahmadi.  Covariance Estimation: The GLM and Regularization Perspectives , 2011, 1202.1661.

[41]  P. Hansen,et al.  A Forecast Comparison of Volatility Models: Does Anything Beat a Garch(1,1)? , 2004 .

[42]  A. P. Dawid,et al.  Probability, Causality and the Empirical World: A Bayes-de Finetti-Popper-Borel Synthesis , 2004 .

[43]  D. Freedman.  On Bootstrapping Two-Stage Least-Squares Estimates in Stationary Linear Models , 1984 .

[44]  Dimitris N. Politis,et al.  Model-free model-fitting and predictive distributions , 2010 .

[45]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[46]  Snigdhansu Chatterjee,et al.  © Institute of Mathematical Statistics, 2005 GENERALIZED BOOTSTRAP FOR ESTIMATING EQUATIONS , 2022 .

[47]  Robert A. Stine,et al.  Estimating Properties of Autoregressive Forecasts , 1987 .

[48]  Seongbaek Yi,et al.  One-Sided Cross-Validation , 1998 .

[49]  D. Tjøstheim,et al.  Nonparametric Estimation and Identification of Nonlinear ARCH Time Series Strong Convergence and Asymptotic Normality: Strong Convergence and Asymptotic Normality , 1995, Econometric Theory.

[50]  V. Yohai,et al.  Robust Statistics: Theory and Methods , 2006 .

[51]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[52]  Arif Dowla.  Locally Stationary Processes and the Local Block Bootstrap , 2003 .

[53]  Galit Shmueli,et al.  To Explain or To Predict? , 2010 .

[54]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[55]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[56]  Efstathios Paparoditis,et al.  Bootstrap methods for dependent data: A review , 2011 .

[57]  W. Härdle,et al.  Bootstrapping in Nonparametric Regression: Local Adaptive Smoothing and Confidence Bands , 1988 .

[58]  A. Antoniadis,et al.  A functional wavelet–kernel approach for time series prediction , 2006 .

[59]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[60]  M. Priestley.  Evolutionary Spectra and Non‐Stationary Processes , 1965 .

[61]  R. R. Hocking.  The analysis and selection of variables in linear regression , 1976 .

[62]  D. Politis,et al.  Bootstrap confidence intervals in nonparametric regression with built-in bias correction , 2008 .

[63]  C. Granger,et al.  Forecasting Volatility in Financial Markets: A Review , 2003 .

[64]  R. Stine.  Bootstrap Prediction Intervals for Regression , 1985 .

[65]  T. Bollerslev,et al.  Generalized autoregressive conditional heteroskedasticity , 1986 .

[66]  M. Bartlett.  On the Theoretical Specification and Sampling Properties of Autocorrelated Time‐Series , 1946 .

[67]  J. Shao,et al.  The jackknife and bootstrap , 1996 .

[68]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[69]  S. Geer,et al.  Regularization in statistics , 2006 .

[70]  Richard A. Davis,et al.  Simple consistent estimation of the coefficients of a linear filter , 1988 .

[71]  Peter Hall,et al.  A Geometrical Method for Removing Edge Effects from Kernel-Type Nonparametric Regression Estimators , 1991 .

[72]  W. Wu,et al.  Covariance and precision matrix estimation for high-dimensional time series , 2013, 1401.0993.

[73]  Zhou Zhou,et al.  Simultaneous inference of linear models with time varying coefficients , 2010 .

[74]  W. Härdle,et al.  Kernel regression smoothing of time series , 1992 .

[75]  Bootstrap Joint Prediction Regions , 2015 .

[76]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[77]  Joseph P. Romano,et al.  The stationary bootstrap , 1994 .

[78]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[79]  M. Pourahmadi,et al.  BANDING SAMPLE AUTOCOVARIANCE MATRICES OF STATIONARY PROCESSES , 2009 .

[80]  Jörg Polzehl,et al.  Simultaneous bootstrap confidence bands in nonparametric regression , 1998 .

[81]  Dimitris N. Politis,et al.  Computer-intensive methods in statistical analysis , 1998, IEEE Signal Process. Mag..

[82]  Jianqing Fan.  Local Linear Regression Smoothers and Their Minimax Efficiencies , 1993 .

[83]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[84]  J. Zakoian,et al.  GARCH Models: Structure, Statistical Inference and Financial Applications , 2010 .

[85]  Efstathios Paparoditis,et al.  A MARKOVIAN LOCAL RESAMPLING SCHEME FOR NONPARAMETRIC ESTIMATORS IN TIME SERIES ANALYSIS , 2001, Econometric Theory.

[86]  Oliver Linton,et al.  An analysis of transformations for additive nonparametric regression , 1997 .

[87]  Frédéric Ferraty,et al.  Nonparametric Functional Data Analysis: Theory and Practice (Springer Series in Statistics) , 2006 .

[88]  Thomas J. Harris,et al.  The use of simplified or misspecified models : Linear case , 2008 .

[89]  Efstathios Paparoditis,et al.  Local block bootstrap , 2002 .

[90]  L. Devroye.  Laws of the Iterated Logarithm for Order Statistics of Uniform Spacings , 1981 .

[91]  Jorma Rissanen,et al.  Properties of infinite covariance matrices and stability of optimum predictors , 1969, Inf. Sci..

[92]  D. Politis.  Adaptive bandwidth choice , 2003 .

[93]  BOOTSTRAP CONFIDENCE INTERVALS FOR CONDITIONAL QUANTILE FUNCTIONS , 1988 .

[94]  N. Altman.  An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[95]  J. Horowitz.  Bootstrap Methods for Median Regression Models , 1996 .

[96]  D. Freedman.  Bootstrapping Regression Models , 1981 .

[97]  J. Hahn.  Bootstrapping Quantile Regression Estimators , 1995, Econometric Theory.

[98]  D. Politis,et al.  Banded and tapered estimates for autocovariance matrices and the linear process bootstrap , 2010 .

[99]  Jagdish K. Patel,et al.  Prediction intervals - a review , 1989 .

[100]  Wolfgang Härdle,et al.  On Bootstrapping Kernel Spectral Estimates , 1992 .

[101]  Asger Lunde,et al.  Choosing the Best Volatility Models: The Model Confidence Set Approach , 2003 .

[102]  H. White,et al.  ASYMPTOTIC DISTRIBUTION THEORY FOR NONPARAMETRIC ENTROPY MEASURES OF SERIAL DEPENDENCE , 2005 .

[103]  Juan Romo,et al.  Bootstrap predictive inference for ARIMA processes , 2004 .

[104]  Lori A. Thombs,et al.  Bootstrap Prediction Intervals for Autoregression , 1990 .

[105]  Andrew J. Patton,et al.  Volatility Forecast Evaluation and Comparison Using Imperfect Volatility Proxies , 2005 .

[106]  Piotr Fryzlewicz,et al.  Forecasting non-stationary time series by wavelet process modelling , 2003 .

[107]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[108]  W. R. Schucany.  Kernel Smoothers: An Overview of Curve Estimators for the First Graduate Course in Nonparametric Statistics , 2004 .

[109]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[110]  J. Wolfowitz.  The Minimum Distance Method , 1957 .

[111]  D. Politis,et al.  NONLINEARITY OF ARCH AND STOCHASTIC VOLATILITY MODELS AND BARTLETT'S FORMULA , 2011 .

[112]  BANDWIDTH SELECTION IN KERNEL SMOOTHING OF TIME SERIES , 1996 .

[113]  J. Robins,et al.  Distribution-Free Prediction Sets , 2013, Journal of the American Statistical Association.

[114]  P. J. Huber.  Robust Regression: Asymptotics, Conjectures and Monte Carlo , 1973 .

[115]  D. Politis,et al.  Rejoinder—Bootstrap prediction intervals for linear, nonlinear and nonparametric autoregressions , 2016 .

[116]  Han Xiao,et al.  Covariance matrix estimation for stationary time series , 2011, 1105.4563.

[117]  R. Beran.  Calibrating Prediction Regions , 1990 .

[118]  Comparison of bootstrap and jackknife variance estimators in linear regression: second order results , 2002 .

[119]  A. Ullah,et al.  Nonparametric Econometrics: Semiparametric and Nonparametric Estimation of Simultaneous Equation Models , 1999 .

[120]  Daniel B. Nelson.  CONDITIONAL HETEROSKEDASTICITY IN ASSET RETURNS: A NEW APPROACH , 1991 .

[121]  Zhou Zhou,et al.  Local linear quantile estimation for nonstationary time series , 2009, 0908.3576.

[122]  A. Bose.  Edgeworth correction by bootstrap in autoregressions , 1988 .

[123]  D. Politis,et al.  High-dimensional autocovariance matrices and optimal linear prediction , 2015 .

[124]  Rodney C. Wolff,et al.  Methods for estimating a conditional distribution function , 1999 .

[125]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[126]  Thomas Mikosch,et al.  Changes of structure in financial time series and the GARCH model , 2004 .

[127]  L. Bachelier,et al.  Theory of Speculation , 1964 .

[128]  E. Paparoditis,et al.  The Hybrid Wild Bootstrap for Time Series , 2012 .

[129]  C. Ing,et al.  Inverse moment bounds for sample autocovariance matrices based on detrended time series and their applications , 2015 .

[130]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[131]  Carsten Jentsch,et al.  Covariance matrix estimation and linear process bootstrap for multivariate time series of possibly increasing dimension , 2015, 1506.00816.

[132]  Dimitris N. Politis,et al.  A normalizing and variance–stabilizing transformation for financial time series , 2003 .

[133]  D. Politis.  Bootstrap Confidence Intervals in Nonparametric Regression Without an Additive Model , 2014 .

[134]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[135]  M. B. Rajarshi.  Bootstrap in Markov-sequences based on estimates of transition density , 1990 .

[136]  R. Chou,et al.  ARCH modeling in finance: A review of the theory and empirical evidence , 1992 .

[137]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[138]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[139]  P. Hall.  On Edgeworth Expansion and Bootstrap Confidence Bands in Nonparametric Curve Estimation , 1993 .

[140]  Alan J. Lee,et al.  Linear Regression Analysis: Seber/Linear , 2003 .

[141]  Joseph P. Romano,et al.  A General Resampling Scheme for Triangular Arrays of $\alpha$-Mixing Random Variables with Application to the Problem of Spectral Density Estimation , 1992 .

[142]  Piotr Kokoszka,et al.  Change-point estimation in ARCH models , 2000 .

[143]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[144]  L. Bachelier,et al.  Théorie de la spéculation , 1900 .

[145]  E. Mammen,et al.  Bootstrap of kernel smoothing in nonlinear time series , 2002 .

[146]  C. Gouriéroux.  ARCH Models and Financial Applications , 1997 .

[147]  N. Draper,et al.  Applied Regression Analysis: Draper/Applied Regression Analysis , 1998 .

[148]  T. Bollerslev,et al.  Analytical Evaluation of Volatility Forecasts , 2002 .

[149]  V. Fabian.  Polynomial Estimation of Regression Functions with the Supremum Norm Error , 1988 .

[150]  J. Friedman,et al.  Estimating Optimal Transformations for Multiple Regression and Correlation. , 1985 .

[151]  Qi Li.  Nonparametric econometrics , 2006 .

[152]  Stefan Sperlich,et al.  Simple and effective boundary correction for kernel densities and regression with an application to the world income and Engel curve estimation , 2010, Comput. Stat. Data Anal..

[153]  S. N. Lahiri,et al.  A necessary and sufficient condition for asymptotic independence of discrete Fourier transforms under short- and long-range dependence , 2003 .

[154]  Juan Romo,et al.  Forecasting time series with sieve bootstrap , 2002 .

[155]  Richard A. Davis,et al.  IMPROVED BOOTSTRAP PREDICTION INTERVALS FOR AUTOREGRESSIONS , 1995 .

[156]  David J. Olive.  Prediction intervals for regression models , 2007, Comput. Stat. Data Anal..

[157]  Dimitris N. Politis,et al.  A heavy-tailed distribution for ARCH residuals with application to volatility prediction , 2004 .

[158]  E. Fama.  The Behavior of Stock-Market Prices , 1965 .

[159]  F. Hampel.  Robust estimation: A condensed partial survey , 1973 .

[160]  Gennady Samorodnitsky,et al.  How misleading can sample ACF's of stable MA's Be? (Very!) , 1998 .

[161]  H. Künsch.  The Jackknife and the Bootstrap for General Stationary Observations , 1989 .

[162]  Thomas L. Vincent,et al.  Pest Management Programs via Optimal Control Theory , 1975 .

[163]  P. Hall.  The Bootstrap and Edgeworth Expansion , 1992 .

[164]  I. Keilegom,et al.  Estimation of a semiparametric transformation model , 2008, 0804.0719.

[165]  R. Dahlhaus.  Locally Stationary Processes , 2011, 1109.4174.

[166]  Jeffrey S. Spence,et al.  Far Casting Cross-Validation , 2009 .

[167]  M. Wand.  Local Regression and Likelihood , 2001 .

[168]  P. Bickel,et al.  Banded regularization of autocovariance matrices in application to parameter estimation and forecasting of time series , 2011 .

[169]  Irène Gijbels,et al.  Understanding exponential smoothing via kernel regression , 1999 .

[170]  Jianqing Fan,et al.  Nonlinear Time Series : Nonparametric and Parametric Methods , 2005 .

[171]  S. Weisberg.  Plots, transformations, and regression , 1985 .

[172]  Claudia Kirch,et al.  TFT-bootstrap: Resampling time series in the frequency domain to obtain replicates in the time domain , 2011, 1211.4732.

[173]  G. Box.  Science and Statistics , 1976 .

[174]  R. Schmoyer.  Asymptotically valid prediction intervals for linear models , 1992 .

[175]  Wei Biao Wu,et al.  Testing parametric assumptions of trends of a nonstationary time series , 2011 .

[176]  R. Engle.  Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[177]  David Ruppert,et al.  Prediction and tolerance intervals with transformation and/or weighting , 1991 .

[178]  Dimitris N. Politis,et al.  Model-Free Versus Model-Based Volatility Prediction , 2007 .

[179]  W. Härdle.  Applied Nonparametric Regression , 1992 .

[180]  M. B. Priestley,et al.  Non-linear and non-stationary time series analysis , 1990 .

[181]  B. Mandelbrot.  The Variation of Certain Speculative Prices , 1963 .