Unbiased Estimation of the Hessian for Partially Observed Diffusions

In this article we consider the development of unbiased estimators of the Hessian, of the loglikelihood function with respect to parameters, for partially observed diffusion processes. These processes arise in numerous applications, where such diffusions require derivative information, either through the Jacobian or Hessian matrix. As time-discretizations of diffusions induce a bias, we provide an unbiased estimator of the Hessian. This is based on using Girsanov’s Theorem and randomization schemes developed through Mcleish [25] and Rhee & Glynn [27]. We demonstrate our developed estimator of the Hessian is unbiased, and one of finite variance. We numerically test and verify this by comparing the methodology here to that of a newly proposed particle filtering methodology. We test this on a range of diffusion models, which include different Ornstein–Uhlenbeck processes and the Fitzhugh–Nagumo model, arising in neuroscience.

[1]  G. Roberts,et al.  Exact simulation of diffusions , 2005, math/0602523.

[2]  Moritz Schauer,et al.  Bayesian estimation of incompletely observed diffusions , 2016, 1606.04082.

[3]  P. Fearnhead,et al.  Particle filters for partially observed diffusions , 2007, 0710.4245.

[4]  Susanne Ditlevsen,et al.  Hypoelliptic diffusions: filtering and inference from complete and partial observations , 2018, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[5]  Jorge Nocedal,et al.  On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning , 2011, SIAM J. Optim..

[6]  S. Shreve Stochastic Calculus for Finance II: Continuous-Time Models , 2010 .

[7]  Peter W. Glynn,et al.  Exact estimation for Markov chain equilibrium expectations , 2014, Journal of Applied Probability.

[8]  Don McLeish,et al.  A general method for debiasing a Monte Carlo estimator , 2010, Monte Carlo Methods Appl..

[9]  M. Andrew,et al.  Stationarity , 2020, Applied Quantitative Analysis for Real Estate.

[10]  Kody J. H. Law,et al.  On Unbiased Estimation for Discretized Models , 2021, SIAM/ASA J. Uncertain. Quantification.

[11]  J. Bierkens,et al.  Simulation of elliptic and hypo-elliptic conditional diffusions , 2018, Advances in Applied Probability.

[12]  J. Blanchet,et al.  Exact simulation for multivariate Itô diffusions , 2017, Advances in Applied Probability.

[13]  M. Rosenbaum,et al.  Volatility is rough , 2014, 1410.3394.

[14]  Yan Zhou,et al.  Bayesian Static Parameter Estimation for Partially Observed Diffusions via Multilevel Monte Carlo , 2017, SIAM J. Sci. Comput..

[15]  Naman Agarwal,et al.  Second-Order Stochastic Optimization for Machine Learning in Linear Time , 2016, J. Mach. Learn. Res..

[16]  P. Fearnhead,et al.  Random‐weight particle filtering of continuous time processes , 2010 .

[17]  Ajay Jasra,et al.  On Unbiased Score Estimation for Partially Observed Diffusions , 2021, 2105.04912.

[18]  Jim Gatheral The Volatility Surface: A Practitioner's Guide , 2006 .

[19]  Andrew J. Majda,et al.  Nonlinear Dynamics and Statistical Theories for Basic Geophysical Flows , 2006 .

[20]  Christophe Andrieu,et al.  Uniform ergodicity of the iterated conditional SMC and geometric ergodicity of particle Gibbs samplers , 2013, 1312.6432.

[21]  J. Hammersley,et al.  Diffusion Processes and Related Topics in Biology , 1977 .

[22]  Harry van Zanten,et al.  Guided proposals for simulating multi-dimensional diffusion bridges , 2013, 1311.3606.

[23]  Matti Vihola,et al.  Unbiased Inference for Discretely Observed Hidden Markov Model Diffusions , 2018, SIAM/ASA J. Uncertain. Quantification.

[24]  D. Crisan,et al.  Fundamentals of Stochastic Filtering , 2008 .

[25]  H. Thorisson Coupling, stationarity, and regeneration , 2000 .

[26]  G. Roberts,et al.  Retrospective exact simulation of diffusion sample paths with applications , 2006 .

[27]  Fredrik Lindsten,et al.  Smoothing With Couplings of Conditional Particle Filters , 2017, Journal of the American Statistical Association.

[28]  Haikady N. Nagaraja,et al.  Inference in Hidden Markov Models , 2006, Technometrics.

[29]  Matti Vihola,et al.  Unbiased Estimators and Multilevel Monte Carlo , 2015, Oper. Res..

[30]  Peter W. Glynn,et al.  Unbiased Estimation with Square Root Convergence for SDE Models , 2015, Oper. Res..

[31]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[32]  T. Faniran Numerical Solution of Stochastic Differential Equations , 2015 .