Particle metropolis hastings using Langevin dynamics

Particle Markov Chain Monte Carlo (PMCMC) samplers allow for routine inference of parameters and states in challenging nonlinear problems. A common choice for the parameter proposal is a simple random walk sampler, which can scale poorly with the number of parameters. In this paper, we propose to use log-likelihood gradients, i.e. the score, in the construction of the proposal, akin to the Langevin Monte Carlo method, but adapted to the PMCMC framework. This can be thought of as a way to guide a random walk proposal by using drift terms that are proportional to the score function. The method is successfully applied to a stochastic volatility model and the drift term exhibits intuitive behaviour.

[1]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[2]  Mikkel N. Schmidt.  Function factorization using warped Gaussian processes , 2009, ICML '09.

[3]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[4]  A. Doucet,et al.  Efficient Bayesian Inference for Switching State-Space Models using Discrete Particle Markov Chain Monte Carlo Methods , 2010, 1011.2437.

[5]  H. Ishwaran.  Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks , 1999 .

[6]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[7]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[8]  Fredrik Lindsten,et al.  On the use of backward simulation in the particle Gibbs sampler , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[10]  Fredrik Lindsten,et al.  Ancestor Sampling for Particle Gibbs , 2012, NIPS.

[11]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[12]  M. Beaumont.  Estimation of population growth or decline in genetically monitored populations. , 2003, Genetics.

[13]  Ralph S. Silva,et al.  On Some Properties of Markov Chain Monte Carlo Simulation Methods Based on the Particle Filter , 2012 .

[14]  A. Kennedy.  The theory of hybrid stochastic algorithms , 1990 .

[15]  Creutz.  Global Monte Carlo algorithms for many-fermion systems. , 1988, Physical review. D, Particles and fields.

[16]  Noelle Foreshaw.  Options… , 2010 .

[17]  L. Dousset.  Understanding Human Relations (Kinship Systems) , 2011 .

[18]  A. Doucet,et al.  Parameter estimation in general state-space models using particle methods , 2003 .

[19]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[20]  D. Pierre.  Forward Smoothing Using Sequential Monte Carlo , 2009 .

[21]  Sumeetpal S. Singh,et al.  Novel particle filter methods for recursive and batch maximum likelihood parameter estimation in general states space models , 2005 .

[22]  S. Chib.  Likelihood based inference for diffusion driven state space models , 2006 .

[23]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[24]  S. Duane,et al.  Hybrid Monte Carlo , 1987 .

[25]  J. Hull.  Options, Futures, and Other Derivatives , 1989 .

[26]  W. Marsden.  I and J , 2012 .

[27]  Thomas B. Schön,et al.  Estimation of general nonlinear state-space systems , 2010, 49th IEEE Conference on Decision and Control (CDC).

[28]  Simon J. Godsill,et al.  Improved Particle Approximations to the Joint Smoothing Distribution Using Markov Chain Monte Carlo , 2013, IEEE Transactions on Signal Processing.

[29]  Aurélien Garivier,et al.  Sequential Monte Carlo smoothing for general state space hidden Markov models , 2011, 1202.2945.

[30]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[31]  F Gustafsson,et al.  Particle filter theory and practice with positioning applications , 2010, IEEE Aerospace and Electronic Systems Magazine.

[32]  Sumeetpal S. Singh,et al.  Particle approximations of the score and observed information matrix in state space models with application to parameter estimation , 2011 .

[33]  A. Doucet,et al.  Monte Carlo Smoothing for Nonlinear Time Series , 2004, Journal of the American Statistical Association.

[34]  Jun S. Liu,et al.  Monte Carlo strategies in scientific computing , 2001 .

[35]  Matthew Charles Higgs,et al.  Approximate inference for state-space models , 2011 .

[36]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.