Penalised Complexity Priors for Stationary Autoregressive Processes

The autoregressive process of order $p$ (AR($p$)) is a central model in time series analysis. A Bayesian approach requires the user to define a prior distribution for the coefficients of the AR($p$) model. Although it is easy to write down some prior, it is not at all obvious how to understand and interpret the prior, to ensure that it behaves according to the users prior knowledge. In this paper, we approach this problem using the recently developed ideas of penalised complexity (PC) priors. These priors have important properties like robustness and invariance to reparameterisations, as well as a clear interpretation. A PC prior is computed based on specific principles, where model component complexity is penalised in terms of deviation from simple base model formulations. In the AR(1) case, we discuss two natural base model choices, corresponding to either independence in time or no change in time. The latter case is illustrated in a survival model with possible time-dependent frailty. For higher-order processes, we propose a sequential approach, where the base model for AR($p$) is the corresponding AR($p-1$) model expressed using the partial autocorrelations. The properties of the new prior are compared with the reference prior in a simulation study.

[1]  A. Gelfand,et al.  High-Resolution Space–Time Ozone Modeling for Assessing Trends , 2007, Journal of the American Statistical Association.

[2]  P. A. Blight The Analysis of Time Series: An Introduction , 1991 .

[3]  David N. DeJong,et al.  Reconsidering 'Trends and random walks in macroeconomic time series' * , 1991 .

[4]  J. Bernardo,et al.  THE FORMAL DEFINITION OF REFERENCE PRIORS , 2009, 0904.0156.

[5]  A. Zellner An Introduction to Bayesian Inference in Econometrics , 1971 .

[6]  J. Monahan A note on enforcing stationarity in autoregressive-moving average models , 1984 .

[7]  R. Kohn,et al.  Bayesian estimation of an autoregressive model using Markov chain Monte Carlo , 1996 .

[8]  C. Mcgilchrist,et al.  ML and REML estimation in survival analysis with time dependent correlated frailty. , 1998, Statistics in medicine.

[9]  Arnold Zellner,et al.  An Introduction to Bayesian Inference in Econometrics. , 1974 .

[10]  Khandoker Shuvo Bakar,et al.  Hierarchical Bayesian autoregressive models for large space–time data with applications to ozone concentration modelling , 2012 .

[11]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[12]  H. Rue,et al.  Scaling intrinsic Gaussian Markov random field priors in spatial modelling , 2014 .

[13]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[14]  L. Fahrmeir,et al.  Multivariate statistical modelling based on generalized linear models , 1994 .

[15]  James O. Berger,et al.  Noninformative Priors and Bayesian Testing for the AR(1) Model , 1994, Econometric Theory.

[16]  Gene H. Golub,et al.  Matrix computations , 1983 .

[17]  B. Liseo,et al.  Objective priors for causal AR(p) with partial autocorrelations , 2013 .

[18]  M. West Kernel density estimation and marginalization consistency , 1991 .

[19]  Michael A. West,et al.  Time Series: Modeling, Computation, and Inference , 2010 .

[20]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[21]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[22]  Christian P. Robert,et al.  The Bayesian choice , 1994 .

[23]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[24]  Finn Lindgren,et al.  Bayesian computing with INLA: New features , 2012, Comput. Stat. Data Anal..

[25]  Eric R. Ziegel,et al.  Multivariate Statistical Modelling Based on Generalized Linear Models , 2002, Technometrics.

[26]  C. Sims MACROECONOMICS AND REALITY , 1977 .

[27]  S. Chib,et al.  Bayes inference via Gibbs sampling of autoregressive time series subject to Markov mean and variance shifts , 1993 .

[28]  Chris Chatfield,et al.  The Analysis of Time Series: An Introduction , 1981 .

[29]  Peter C. B. Phillips,et al.  To Criticize the Critics: An Objective Bayesian Analysis of Stochastic Trends , 1991 .

[30]  Douglas M. Hawkins,et al.  The Problem of Overfitting , 2004, J. Chem. Inf. Model..

[31]  O. Barndorff-Nielsen,et al.  On the parametrization of autoregressive models by partial autocorrelations , 1973 .

[32]  Siddhartha Chib,et al.  Bayes regression with autoregressive errors : A Gibbs sampling approach , 1993 .

[33]  D. Harrington,et al.  Counting Processes and Survival Analysis , 1991 .

[34]  Thiago G. Martins,et al.  Penalising Model Component Complexity: A Principled, Practical Approach to Constructing Priors , 2014, 1403.4630.

[35]  J. LeSage Bayesian Estimation of Spatial Autoregressive Models , 1997 .

[36]  D. Harrington,et al.  Counting Processes and Survival Analysis: Fleming/Counting , 2005 .

[37]  Renate Meyer,et al.  Bayesian inference for recurrent events data using time-dependent frailty. , 2005, Statistics in medicine.

[38]  H. Rue,et al.  Approximate Bayesian Inference for Survival Models , 2010 .