Linearization Methods in Time Series Analysis

In this dissertation, we propose a set of computationally efficient methods based on approximating/representing nonlinear processes by linear ones, so-called linearization. Firstly, a linearization method is introduced for estimating the multiple frequencies in sinusoidal processes. It utilizes a regularized autoregressive (AR) approximation, which can be regarded as a “large p small n” approach in a time series context. An appealing property of regularized AR is that it avoids a model selection step and allows for an efficient updating of the frequency estimates whenever new observations are obtained. The theoretical analysis shows that the regularized AR frequency estimates are consistent and asymptotically normally distributed. Secondly, a sieve bootstrap scheme is proposed using the linear representation of generalized autoregressive conditional heteroscedastic (GARCH) models to construct prediction intervals (PIs) for the returns and volatilities. Our method is simple, fast and distribution-free, while providing sharp and wellcalibrated PIs. A similar linear bootstrap scheme can also be used for diagnostic testing. Thirdly, we introduce a robust lagrange multiplier (LM) test, which utilizes either the bootstrap or permutation procedure to obtain critical values, for detecting GARCH effects. We justify that both bootstrap and permutation LM tests are consistent. Intensive numerical studies indicate that the proposed resampling algorithms significantly improve the size and power of the LM test in both skewed and heavy-tailed processes. Moreover, fourthly, we introduce a nonparametric trend test in the presence of GARCH effects (NT-GARCH) based on heteroscedastic ANOVA. Our empirical evidence show that NT-GARCH can effectively detect non-monotonic trends under GARCH, especially in the presence of irregular seasonal components. We suggest to apply the bootstrap procedure for both selecting the window length and finding critical values. The newly proposed methods are illustrated by applications to astronomical data, to foreign currency exchange rates

[1]  R. Engle Dynamic Conditional Correlation , 2002 .

[2]  T. Bollerslev,et al.  Generalized autoregressive conditional heteroskedasticity , 1986 .

[3]  M. S. Mackisack,et al.  SOME PROPERTIES OF AUTOREGRESSIVE ESTIMATES FOR PROCESSES WITH MIXED SPECTRA , 1990 .

[4]  N. Watanabe,et al.  A fuzzy trend model for long-term financial time series and its identification , 2006, NAFIPS 2006 - 2006 Annual Meeting of the North American Fuzzy Information Processing Society.

[5]  S. Weisberg,et al.  Diagnostics for heteroscedasticity in regression , 1983 .

[6]  Anil K. Bera,et al.  ARCH Models: Properties, Estimation and Testing , 1993 .

[7]  Hui-Chen Su,et al.  A nonlinear time series analysis using two‐stage genetic algorithms for streamflow forecasting , 2008 .

[8]  H. Akaike A new look at the statistical model identification , 1974 .

[9]  P. Doukhan Mixing: Properties and Examples , 1994 .

[10]  Ta-Hsin Li,et al.  Asymptotic normality of sample autocovariances with an application in frequency estimation , 1994 .

[11]  Hideaki Sakai,et al.  Statistical analysis of Pisarenko's method for sinusoidal frequency estimation , 1984 .

[12]  Juan Romo,et al.  Bootstrap predictive inference for ARIMA processes , 2004 .

[13]  Lori A. Thombs,et al.  Bootstrap Prediction Intervals for Autoregression , 1990 .

[14]  Daniel B. Nelson Stationarity and Persistence in the GARCH(1,1) Model , 1990, Econometric Theory.

[15]  Ta-Hsin Li,et al.  Tracking abrupt frequency changes , 1998 .

[16]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  T. Hockey Galileo's Planet: Observing Jupiter Before Photography , 1998 .

[18]  Beum-Jo Park,et al.  An outlier robust GARCH model and forecasting volatility of exchange rate returns , 2002 .

[19]  Wen Wang,et al.  Testing and modelling autoregressive conditional heteroskedasticity of streamflow processes , 2005 .

[20]  A. Bose Edgeworth correction by bootstrap in autoregressions , 1988 .

[21]  Robert Andersen Modern Methods for Robust Regression , 2007 .

[22]  M. Rosenblatt Remarks on a Multivariate Transformation , 1952 .

[23]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[24]  A. Barabanov,et al.  Convergence of the least-squares method with a polynomial regularizer for the infinite-dimensional autoregression equation , 2005 .

[25]  H. Wong,et al.  Portmanteau test for conditional heteroscedasticity, using ranks of squared residuals , 1995 .

[26]  A. Barabanov,et al.  Strong consistency of the regularized least-squares estimates of infinite autoregressive models , 2007 .

[27]  A. M. Walker On the estimation of a harmonic component in a time series with stationary independent residuals , 1971 .

[28]  P. Bühlmann Sieve bootstrap for time series , 1997 .

[29]  P. Bougerol,et al.  Strict Stationarity of Generalized Autoregressive Processes , 1992 .

[30]  J. J. Reeves Bootstrap prediction intervals for ARCH models , 2005 .

[31]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[32]  Joseph P. Romano,et al.  Inference for Autocorrelations under Weak Assumptions , 1996 .

[33]  Y. Gel,et al.  Regularized Autoregressive Multiple Frequency Estimation , 2011 .

[34]  Langford B. White,et al.  Asymptotic statistical properties of AR Spectral estimators for processes with mixed spectra , 2002, IEEE Trans. Inf. Theory.

[35]  W. Härdle,et al.  Bootstrap Methods for Time Series , 2003 .

[36]  V. Pisarenko The Retrieval of Harmonics from a Covariance Function , 1973 .

[37]  Khaled H. Hamed,et al.  A modified Mann-Kendall trend test for autocorrelated data , 1998 .

[38]  Barry G. Quinn On Kay's frequency estimator , 2000 .

[39]  Juan Romo,et al.  Bootstrap prediction for returns and volatilities in GARCH models , 2006, Comput. Stat. Data Anal..

[40]  T. Breurch,et al.  A simple test for heteroscedasticity and random coefficient variation (econometrica vol 47 , 1979 .

[41]  D. Brillinger Fitting Cosines: Some Procedures and Some Physical Examples , 1987 .

[42]  D. Politis The Impact of Bootstrap Methods on Time Series Analysis , 2003 .

[43]  Ching-Zong Wei,et al.  Order selection for same-realization predictions in autoregressive processes , 2005, math/0602326.

[44]  E. J. Hannan,et al.  ON‐LINE FREQUENCY ESTIMATION , 1993 .

[45]  D. Freedman,et al.  Some Asymptotic Theory for the Bootstrap , 1981 .

[46]  Andrew J. Patton,et al.  What good is a volatility model? , 2001 .

[47]  K. Berk Consistent Autoregressive Spectral Estimates , 1974 .

[48]  Tim Bollerslev,et al.  Glossary to ARCH (GARCH) , 2008 .

[49]  E. Hannan,et al.  Autocorrelation, Autoregression and Autoregressive Approximation , 1982 .

[50]  R. Leipus,et al.  STATIONARY ARCH MODELS: DEPENDENCE STRUCTURE AND CENTRAL LIMIT THEOREM , 2000, Econometric Theory.

[51]  P Darania,et al.  LINEARIZATION METHOD FOR SOLVING NONLINEAR INTEGRAL EQUATIONS , 2006 .

[52]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[53]  N. Balakrishnan,et al.  A nonparametric test for trend based on initial ranks , 2006 .

[54]  Ronald K. Pearson,et al.  BMC Bioinformatics BioMed Central Methodology article , 2005 .

[55]  Oliver Linton,et al.  A CLOSED-FORM ESTIMATOR FOR THE GARCH(1,1) MODEL , 2006, Econometric Theory.

[56]  A. I. McLeod,et al.  DIAGNOSTIC CHECKING ARMA TIME SERIES MODELS USING SQUARED‐RESIDUAL AUTOCORRELATIONS , 1983 .

[57]  R. Kumaresan,et al.  Estimation of frequencies of multiple sinusoids: Making linear prediction perform like maximum likelihood , 1982, Proceedings of the IEEE.

[58]  W. Li,et al.  Detecting and Diagnostic Checking Multivariate Conditional Heteroscedastic Time Series Models , 2002 .

[59]  W. Li,et al.  Diagnostic checking of nonlinear multivariate time series with multivariate arch errors , 1997 .

[60]  Bonnie K. Ray,et al.  Model selection and forecasting for long‐range dependent processes , 1996 .

[61]  P. Bougerol,et al.  Stationarity of Garch processes and of some nonnegative time series , 1992 .

[62]  Enrique Sentana,et al.  Testing for GARCH effects: a one-sided approach , 1998 .

[63]  Qing Huo Liu,et al.  Generalization of iterative Fourier interpolation algorithms for single frequency estimation , 2011, Digit. Signal Process..

[64]  Christian Léger,et al.  Bootstrap adaptive estimation: The trimmed-mean example , 1990 .

[65]  Weiwen Miao,et al.  Robust directed tests of normality against heavy-tailed alternatives , 2007, Comput. Stat. Data Anal..

[66]  Donald H. Burn,et al.  Hydrologic effects of climatic change in west-central Canada , 1994 .

[67]  Dharmendra Lingaiah,et al.  The Estimation and Tracking of Frequency , 2004 .

[68]  H. Künsch The Jackknife and the Bootstrap for General Stationary Observations , 1989 .

[69]  C. Granger,et al.  AN INTRODUCTION TO LONG‐MEMORY TIME SERIES MODELS AND FRACTIONAL DIFFERENCING , 1980 .

[70]  J. Zakoian,et al.  Estimating linear representations of nonlinear processes , 1998 .

[71]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[72]  M. Akritas,et al.  TWO-WAY HETEROSCEDASTIC ANOVA WHEN THE NUMBER OF LEVELS IS LARGE , 2006 .

[73]  R. Baillie,et al.  Prediction in dynamic models with time-dependent conditional variances , 1992 .

[74]  Bovas Abraham,et al.  Computationally efficient bootstrap prediction intervals for returns and volatilities in ARCH and GARCH processes , 2011 .

[75]  H. Iemoto Modelling the persistence of conditional variances , 1986 .

[76]  On kendall's tau as a test of trend in time series data , 1992 .

[77]  Ta-Hsin Li,et al.  On Convergence and Bias Correction of a Joint Estimation Algorithm for Multiple Sinusoidal Frequencies , 2006 .

[78]  C. Radhakrishna Rao,et al.  Asymptotic behavior of maximum likelihood estimates of superimposed exponential signals , 1993, IEEE Trans. Signal Process..

[79]  E. Parzen Some recent advances in time series modeling , 1974 .

[80]  J. G. Bryan,et al.  STATISTICAL METHODS IN FORECASTING , 1962 .

[81]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[82]  Jesus Miguel,et al.  Bootstrapping forecast intervals in ARCH models , 1999 .

[83]  Piotr Kokoszka,et al.  LARGE SAMPLE DISTRIBUTION OF WEIGHTED SUMS OF ARCH(p) SQUARED RESIDUAL CORRELATIONS , 2001, Econometric Theory.

[84]  Keith W. Hipel,et al.  Data analysis of water quality time series in Lake Erie , 1988 .

[85]  R. Engle Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[86]  D. Burn,et al.  Detection of hydrologic trends and variability , 2002 .

[87]  R. Forthofer,et al.  Rank Correlation Methods , 1981 .

[88]  P. Olver Nonlinear Systems , 2013 .

[89]  Anne Peguin-Feissolle A comparison of the power of some tests for conditional heteroscedasticity , 1999 .

[90]  David F. Hendry,et al.  Small-Sample Properties of ARCH Estimators and Tests , 1985 .

[91]  R. Shibata Asymptotically Efficient Selection of the Order of the Model for Estimating Parameters of a Linear Process , 1980 .

[92]  I. Ibragimov,et al.  Independent and stationary sequences of random variables , 1971 .

[93]  M. Pourahmadi,et al.  Nonparametric estimation of large covariance matrices of longitudinal data , 2003 .

[94]  M. S. Mackisack,et al.  Autoregressive frequency estimation , 1989 .

[95]  Francesca Pianosi,et al.  Flood forecasting for heteroscedastic streamflow processes , 2008 .

[96]  Simulation-based tests for heteroskedasticity: some further results , 2004 .

[97]  I. Keilegom,et al.  Nonparametric Test for the Form of Parametric Regression with Time Series Errors , 2007 .

[98]  P. Bickel,et al.  On the Choice of m in the m Out of n Bootstrap and its Application to Condence Bounds for Extreme Percentiles y , 2005 .

[99]  C. Duguay,et al.  Bootstrap-based tests for trends in hydrological time series, with application to ice phenology data , 2011 .

[100]  K. West,et al.  The Predictive Ability of Several Models of Exchange Rate Volatility , 1994 .

[101]  G. Yule On the Time‐Correlation Problem, with Especial Reference to the Variate‐Difference Correlation Method , 1921 .

[102]  Alan David Hutson,et al.  Resampling Methods for Dependent Data , 2004, Technometrics.

[103]  Rainer Dahlhaus,et al.  Hidden Frequency Estimation with Data Tapers , 2000 .

[104]  B. Kedem,et al.  A note on autocovariance estimation in the presence of discrete spectra , 1995 .

[105]  Wenceslao González-Manteiga,et al.  Saving computer time in constructing consistent bootstrap prediction intervals for autoregressive processes , 1997 .

[106]  Wen Wang Stochasticity, nonlinearity and forecasting of streamflow processes , 2006 .

[108]  Anil K. Bera,et al.  Efficient tests for normality, homoscedasticity and serial independence of regression residuals , 1980 .

[109]  Mario Bertero,et al.  The Stability of Inverse Problems , 1980 .

[110]  Khaled H. Hamed Trend detection in hydrologic data: The Mann–Kendall trend test under the scaling hypothesis , 2008 .

[111]  T. Bollerslev,et al.  ANSWERING THE SKEPTICS: YES, STANDARD VOLATILITY MODELS DO PROVIDE ACCURATE FORECASTS* , 1998 .

[112]  Y. Gel,et al.  Robust Lagrange multiplier test for detecting ARCH/GARCH effect using permutation and bootstrap , 2012 .

[113]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[114]  J. Franke,et al.  BOOTSTRAPPING STATIONARY AUTOREGRESSIVE MOVING-AVERAGE MODELS , 1992 .

[115]  Donald Poskitt,et al.  Autoregressive approximation in nonstandard situations: the fractionally integrated and non-invertible cases , 2007 .

[116]  Petre Stoica,et al.  Asymptotic bias of the high-order autoregressive estimates of sinusoidal frequencies , 1987 .

[117]  Alexander Lindner,et al.  Stationarity, Mixing, Distributional Properties and Moments of GARCH(p, q)-Processes , 2009 .

[118]  E. Hannan The asymptotic theory of linear time-series models , 1973, Journal of Applied Probability.

[119]  Bonnie K. Ray,et al.  MODELING LONG‐MEMORY PROCESSES FOR OPTIMAL LONG‐RANGE PREDICTION , 1993 .

[120]  R. Tibshirani,et al.  The solution path of the generalized lasso , 2010, 1005.1971.

[121]  Eric R. Ziegel,et al.  Analysis of Financial Time Series , 2002, Technometrics.

[122]  Khaled H. Hamed Exact Distribution of the Mann-Kendall Trend Test Statistic for Persistent Data , 2009 .

[123]  Andrew A. Weiss,et al.  Asymptotic Theory for ARCH Models: Estimation and Testing , 1986, Econometric Theory.

[124]  Leslie Godfrey,et al.  Testing for multiplicative heteroskedasticity , 1978 .

[125]  Ser-Huang Poon,et al.  A Practical Guide to Forecasting Financial Market Volatility , 2005 .

[126]  Piotr Kokoszka,et al.  GARCH processes: structure and estimation , 2003 .

[127]  M. Akritas,et al.  Heteroscedastic One-Way ANOVA and Lack-of-Fit Tests , 2004 .

[128]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[129]  Siew Lan Loo Neural networks for financial forecasting , 1994 .

[130]  Christian Francq,et al.  Covariance matrix estimation for estimators of mixing weak ARMA models , 2000 .

[131]  P. Bühlmann Bootstraps for Time Series , 2002 .

[132]  Juan Romo,et al.  On sieve bootstrap prediction intervals , 2003 .

[133]  Allan W. Gregory A Nonparametric Test for Autoregressive Conditional Heteroscedasticity: A Markov-Chain Approach , 1989 .

[134]  Petre Stoica,et al.  Asymptotic properties of the high-order Yule-Walker estimates of sinusoidal frequencies , 1989, IEEE Trans. Acoust. Speech Signal Process..

[135]  Roger Koenker,et al.  A note on studentizing a test for heteroscedasticity , 1981 .

[136]  Joseph P. Romano Bootstrap and randomization tests of some nonparametric hypotheses , 1989 .

[137]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[138]  Regina Y. Liu Moving blocks jackknife and bootstrap capture weak dependence , 1992 .

[139]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[140]  Ching-Kang Ing,et al.  On same-realization prediction in an infinite-order autoregressive process , 2003 .

[141]  Juan Romo,et al.  Forecasting time series with sieve bootstrap , 2002 .

[142]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[143]  Jean-Marie Dufour,et al.  Simulation-based finite-sample tests for heteroskedasticity and ARCH effects , 2004 .

[144]  Michael J. Artis,et al.  Analyzing Strongly Periodic Series in the Frequency Domain: A Comparison of Alternative Approaches with Applications , 2007 .

[145]  A. Tikhonov On the stability of inverse problems , 1943 .

[146]  R. Engle,et al.  The Spline-Garch Model for Low Frequency Volatility and its Global Macroeconomic Causes , 2006 .

[147]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[148]  E. J. Hannan,et al.  Non-linear time series regression , 1971, Journal of Applied Probability.

[149]  M. Wegkamp,et al.  Consistent variable selection in high dimensional regression via multiple testing , 2006 .

[150]  Christian Conrad,et al.  The impulse response function of the long memory GARCH process , 2006 .

[151]  Marcelo C. Medeiros,et al.  MODELING MULTIPLE REGIMES IN FINANCIAL VOLATILITY WITH A FLEXIBLE COEFFICIENT GARCH(1,1) MODEL , 2009, Econometric Theory.

[152]  Peter Bühlmann,et al.  Moving-average representation of autoregressive approximations , 1995 .

[153]  Yulia R. Gel,et al.  Identification of an unstable ARMA equation , 2001 .

[154]  C. Withers Central Limit Theorems for dependent variables. I , 1981 .

[155]  Peter J. Bickel,et al.  A new mixing notion and functional central limit theorems for a sieve bootstrap in time series , 1999 .

[156]  B. Raunig Detecting Arch Effects in Non-Gaussian Time Series , 2007 .

[157]  Roberto López-Valcarce,et al.  Frequency estimation of real-valued single-tone in colored noise using multiple autocorrelation lags , 2010, Signal Process..

[158]  R. Engle Dynamic Conditional Correlation : A Simple Class of Multivariate GARCH Models , 2000 .

[159]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[160]  John H. H. Lee A Lagrange Multiplier Test for Garch Models , 1991 .

[161]  Daniel B. Nelson CONDITIONAL HETEROSKEDASTICITY IN ASSET RETURNS: A NEW APPROACH , 1991 .

[162]  Carlo Novara,et al.  Nonlinear Time Series , 2003 .

[163]  M. Rosenblatt Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[164]  H. Akaike Fitting autoregressive models for prediction , 1969 .

[165]  H. B. Mann Nonparametric Tests Against Trend , 1945 .

[166]  Dawei Huang,et al.  ON LOW AND HIGH FREQUENCY ESTIMATION , 1996 .

[167]  T. Bengtsson,et al.  Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants , 2007 .

[168]  W. Hoeffding The Large-Sample Power of Tests Based on Permutations of Observations , 1952 .

[169]  Juan Toro,et al.  The detection of hidden periodicities: A comparison of alternative methods , 2004 .

[170]  George Sugihara,et al.  Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series , 1990, Nature.

[171]  Stephen E. Fienberg,et al.  Testing Statistical Hypotheses , 2005 .

[172]  Lan Wang,et al.  An ANOVA-type nonparametric diagnostic test for heteroscedastic regression models , 2008 .

[173]  P. Bickel,et al.  Banded regularization of autocovariance matrices in application to parameter estimation and forecasting of time series , 2011 .

[174]  Yulia R. Gel,et al.  Autoregressive frequency detection using Regularized Least Squares , 2010, J. Multivar. Anal..

[175]  J. Gastwirth,et al.  A robust modification of the Jarque–Bera test of normality , 2008 .

[176]  Changshui Zhang,et al.  Multiple Fundamental Frequency Estimation by Modeling Spectral Peaks and Non-Peak Regions , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[177]  F. Diebold,et al.  The Distribution of Realized Exchange Rate Volatility , 2000 .

[178]  N. Shephard Statistical aspects of ARCH and stochastic volatility , 1996 .

[179]  Chen Zhao-Guo AN ALTERNATIVE CONSISTENT PROCEDURE FOR DETECTING HIDDEN FREQUENCIES , 1988 .

[180]  R. Baillie,et al.  Fractionally integrated generalized autoregressive conditional heteroskedasticity , 1996 .