Particle filters and Bayesian inference in financial econometrics

In this paper we review sequential Monte Carlo (SMC) methods, or particle filters (PF), with special emphasis on its potential applications in financial time series analysis and econometrics. We start with the well-known normal dynamic linear model, also known as the normal linear state space model, for which sequential state learning is available in closed form via standard Kalman filter and Kalman smoother recursions. Particle filters are then introduced as a set of Monte Carlo schemes that enable Kalman-type recursions when normality or linearity or both are abandoned. The seminal bootstrap filter (BF) of Gordon, Salmond and Smith (1993) is used to introduce the SMC jargon, potentials and limitations. We also review the literature on parameter learning, an area that started to attract much attention from the particle filter community in recent years. We give particular attention to the Liu–West filter (2001), Storvik filter (2002) and particle learning (PL) of Carvalho, Johannes, Lopes and Polson (2010). We argue that the BF and the auxiliary particle filter (APF) of Pitt and Shephard (1999) define two fundamentally distinct directions within the particle filter literature. We also show that the distinction is more pronounced with parameter learning and argue that PL, which follows the APF direction, is an attractive extension. One of our contributions is to sort out the research from BF to APF (during the 1990s), from APF to now (the 2000s) and from Liu–West filter to Storvik filter to PL. To this end, we provide code in R for all the examples of the paper. Readers are invited to find their own way into this dynamic and active research arena. Copyright © 2010 John Wiley & Sons, Ltd.

[1]  S. J. Koopman.  Discussion of `Particle Markov chain Monte Carlo methods – C. Andrieu, A. Doucet and R. Holenstein’ [Review of: Particle Markov chain Monte Carlo methods] , 2010 .

[2]  Alan E. Gelfand,et al.  Bayesian statistics without tears: A sampling-resampling perspective , 1992 .

[3]  D. Gamerman,et al.  Comparison of Sampling Schemes for Dynamic Linear Models , 2006 .

[4]  Arnaud Doucet,et al.  Particle methods for optimal filter derivative: application to parameter estimation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[5]  N. Shephard,et al.  Stochastic volatility with leverage: Fast and efficient likelihood inference , 2007 .

[6]  Drew D. Creal.  Analysis of filtering and smoothing algorithms for Lévy-driven stochastic volatility models , 2008, Comput. Stat. Data Anal..

[7]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[8]  Christophe Andrieu,et al.  Particle methods for change detection, system identification, and control , 2004, Proceedings of the IEEE.

[9]  Carlos M. Carvalho,et al.  Simulation-based sequential analysis of Markov switching stochastic volatility models , 2007, Comput. Stat. Data Anal..

[10]  Simon J. Godsill,et al.  An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo , 2007, Proceedings of the IEEE.

[11]  Mike K. P. So,et al.  A Stochastic Volatility Model With Markov Switching , 1998 .

[12]  N. Shephard,et al.  Analysis of high dimensional multivariate stochastic volatility models , 2006 .

[13]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[14]  N. Shephard.  Partial non-Gaussian state space , 1994 .

[15]  A. Doucet,et al.  Parameter estimation in general state-space models using particle methods , 2003 .

[16]  O. Cappé,et al.  Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models , 2006, math/0609514.

[17]  Dani Gamerman,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 1997 .

[18]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[19]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo Methods for Computing Bayes Factors , 2001 .

[20]  Nicholas G. Polson,et al.  The Impact of Jumps in Volatility and Returns , 2000 .

[21]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[22]  Michael A. West,et al.  Combined Parameter and State Estimation in Simulation-Based Filtering , 2001, Sequential Monte Carlo Methods in Practice.

[23]  Nicholas G. Polson,et al.  Tracking Flu Epidemics Using Google Flu Trends and Particle Learning , 2009 .

[24]  Francesca Petralia,et al.  Sequential Monte Carlo Estimation of DSGE Models , 2010 .

[25]  Nicholas G. Polson,et al.  Particle Learning and Smoothing , 2010, 1011.1098.

[26]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[27]  H. Sorenson,et al.  Nonlinear Bayesian estimation using Gaussian sum approximations , 1972 .

[28]  Nicholas G. Polson,et al.  Practical filtering with sequential parameter learning , 2008 .

[29]  Nicholas G. Polson,et al.  Optimal Filtering of Jump Diffusions: Extracting Latent States from Asset Prices , 2009 .

[30]  Arnaud Doucet,et al.  Inference for Lévy‐Driven Stochastic Volatility Models via Adaptive Sequential Monte Carlo , 2011 .

[31]  A. Doucet,et al.  A note on auxiliary particle filters , 2008 .

[32]  A. Doucet,et al.  Smoothing algorithms for state–space models , 2010 .

[33]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[34]  Cindy L. Yu,et al.  A Bayesian Analysis of Return Dynamics with Lévy Jumps , 2008 .

[35]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[36]  Christophe Andrieu,et al.  Efficient particle filtering for jump Markov systems. Application to time-varying autoregressions , 2003, IEEE Trans. Signal Process..

[37]  P. Fearnhead,et al.  Particle filters for partially observed diffusions , 2007, 0710.4245.

[38]  Junye Li.  Sequential Bayesian Analysis of Time-Changed Infinite Activity Derivatives Pricing Models , 2009 .

[39]  P. Fearnhead,et al.  Improved particle filter for nonlinear problems , 1999 .

[40]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter , 1990 .

[41]  A. Doucet,et al.  Particle filtering for partially observed Gaussian state space models , 2002 .

[42]  P. Fearnhead,et al.  On‐line inference for hidden Markov models via particle filters , 2003 .

[43]  P. Fearnhead,et al.  Exact filtering for partially observed continuous time models , 2004 .

[44]  P. Fearnhead.  Markov chain Monte Carlo, Sufficient Statistics, and Particle Filters , 2002 .

[45]  Michael A. West,et al.  BAYESIAN MODEL ASSESSMENT IN FACTOR ANALYSIS , 2004 .

[46]  S. Frühwirth-Schnatter.  Data Augmentation and Dynamic Linear Models , 1994 .

[47]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[48]  Jun S. Liu,et al.  Sequential importance sampling for nonparametric Bayes models: The next generation , 1999 .

[49]  Nicholas G. Polson,et al.  A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling , 1992 .

[50]  R. Douc,et al.  Optimality of the auxiliary particle filter , 2009 .

[51]  A. Doucet,et al.  Monte Carlo Smoothing for Nonlinear Time Series , 2004, Journal of the American Statistical Association.

[52]  Jesús Fernández-Villaverde,et al.  Estimating Dynamic Equilibrium Economies: Linear Versus Nonlinear Likelihood , 2004 .

[53]  Nicholas G. Polson,et al.  Particle learning for general mixtures , 2010 .

[54]  N. Shephard,et al.  Markov chain Monte Carlo methods for stochastic volatility models , 2002 .

[55]  M. West,et al.  Dynamic Generalized Linear Models and Bayesian Forecasting , 1985 .

[56]  Silvano Bordignon,et al.  Sequential Monte Carlo Methods for Stochastic Volatility Models with Jumps , 2006 .

[57]  N. Shephard,et al.  Stochastic Volatility: Likelihood Inference And Comparison With Arch Models , 1996 .

[58]  Ruey S. Tsay,et al.  Analysis of Financial Time Series: Tsay/Analysis of Financial Time Series , 2005 .

[59]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[60]  Nicholas G. Polson,et al.  Sequential Parameter Estimation in Stochastic Volatility Models with Jumps , 2006 .

[61]  Robert B. Gramacy,et al.  Particle Learning of Gaussian Process Models for Sequential Design and Optimization , 2009, 0909.5262.

[62]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[63]  N. Chopin.  A sequential particle filter method for static models , 2002 .

[64]  Giovanni Petris,et al.  Dynamic Linear Models with R , 2009 .

[65]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[66]  R. Kohn,et al.  On Gibbs sampling for state space models , 1994 .

[67]  M. West.  Approximating posterior distributions by mixtures , 1993 .

[68]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[69]  Rong Chen,et al.  New sequential Monte Carlo methods for nonlinear dynamic systems , 2005, Stat. Comput..

[70]  Carlos M. Carvalho,et al.  Particle Learning for Sequential Bayesian Computation , 2012 .

[71]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[72]  Yuguo Chen,et al.  Identification and Adaptive Control of Change-Point ARX Models Via Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Automatic Control.

[73]  David N. DeJong,et al.  Effi cient Likelihood Evaluation of State-Space Representations , 2011 .

[74]  Branko Ristic,et al.  Beyond the Kalman Filter: Particle Filters for Tracking Applications , 2004 .

[75]  Kazufumi Ito,et al.  Gaussian filters for nonlinear filtering problems , 2000, IEEE Trans. Autom. Control..

[76]  Jun S. Liu,et al.  Mixture Kalman filters , 2000 .

[77]  Aurélien Garivier,et al.  ON THE FORWARD FILTERING BACKWARD SMOOTHING PARTICLE APPROXIMATIONS OF THE SMOOTHING DISTRIBUTION IN GENERAL STATE SPACES MODELS , 2009, 0904.0316.

[78]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[79]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[80]  Geir Storvik,et al.  Particle filters for state-space models with the presence of unknown static parameters , 2002, IEEE Trans. Signal Process..

[81]  M. West,et al.  Modelling Agent Forecast Distributions , 1992 .

[82]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[83]  Giovanni Petris,et al.  Dynamic linear models , 2009 .

[84]  Jean-Francois Richard,et al.  Efficient Filtering in State-Space Representations , 2008 .

[85]  Siddhartha Chib,et al.  Stochastic Volatility with Leverage: Fast Likelihood Inference , 2004 .

[86]  Paul Fearnhead,et al.  Particle filters for mixture models with an unknown number of components , 2004, Stat. Comput..

[87]  D. Dunson,et al.  Bayesian Variable Selection via Particle Stochastic Search. , 2011, Statistics & probability letters.

[88]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[89]  P. Fearnhead,et al.  A sequential smoothing algorithm with linear computational cost. , 2010 .

[90]  Robert B. Gramacy,et al.  Dynamic Trees for Learning and Design , 2009, 0912.1586.

[91]  Nicholas G. Polson,et al.  Sequential Learning, Predictability, and Optimal Portfolio Returns , 2013 .

[92]  Peter E. Rossi,et al.  Bayesian Analysis of Stochastic Volatility Models , 1994 .

[93]  A. Doucet,et al.  On-Line Parameter Estimation in General State-Space Models , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.