Gaussian Process Approximations of Stochastic Differential Equations

Stochastic differential equations arise naturally in a range of contexts, from financial to environmental modeling. Current solution methods are limited in their representation of the posterior process in the presence of data. In this work, we present a novel Gaussian process approximation to the posterior measure over paths for a general class of stochastic differential equations in the presence of observations. The method is applied to two simple problems: the Ornstein-Uhlenbeck process, of which the exact solution is known and can be compared to, and the double-well system, for which standard approaches such as the ensemble Kalman smoother fail to provide a satisfactory result. Experiments show that our variational approximation is viable and that the results are very promising as the variational approximate solution outperforms standard Gaussian process regression for non-Gaussian Markov processes.

[1]  G. Uhlenbeck,et al.  On the Theory of the Brownian Motion , 1930 .

[2]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[3]  P. Kloeden,et al.  Numerical Solution of Stochastic Differential Equations , 1992 .

[4]  G. J. Haltiner.  Numerical Prediction and Dynamic Meteorology , 1980 .

[5]  J. Elgin.  The Fokker-Planck Equation: Methods of Solution and Applications , 1984 .

[6]  P. Courtier,et al.  A strategy for operational implementation of 4D‐Var, using an incremental approach , 1994 .

[7]  Michael Ghil,et al.  Advanced data assimilation in strongly nonlinear dynamical systems , 1994 .

[8]  G. Evensen,et al.  An ensemble Kalman smoother for nonlinear dynamics , 2000 .

[9]  G. Eyink,et al.  Most Probable Histories for Nonlinear Dynamics: Tracking Climate Transitions , 2000 .

[10]  Tommi S. Jaakkola,et al.  Tutorial on variational approximation methods , 2000 .

[11]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[12]  Lehel Csató,et al.  Sparse On-Line Gaussian Processes , 2002, Neural Computation.

[13]  J. Whitaker,et al.  Ensemble Data Assimilation without Perturbed Observations , 2002 .

[14]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[15]  Andrew C. Lorenc,et al.  The potential of the ensemble Kalman filter for NWP—a comparison with 4D‐Var , 2003 .

[16]  John M. Winn,et al.  Variational Message Passing and its Applications , 2004 .

[17]  Matthias W. Seeger,et al.  Gaussian Processes For Machine Learning , 2004, Int. J. Neural Syst..

[18]  G. Eyink,et al.  A mean field approximation in data assimilation for nonlinear dynamics , 2004 .

[19]  M. Opper,et al.  inverse problems: some new approaches , 2022 .

[20]  S. Dance.  Issues in high resolution limited area data assimilation for quantitative precipitation forecasting , 2004 .

[21]  Christopher K. Wikle,et al.  Atmospheric Modeling, Data Assimilation, and Predictability , 2005, Technometrics.

[22]  R. Seiffert,et al.  Subscale forcing in a global atmospheric circulation model and stochastic parametrization , 2006 .

[23]  Darren J. Wilkinson,et al.  Bayesian sequential inference for nonlinear multivariate diffusions , 2006, Stat. Comput..

[24]  Neil D. Lawrence,et al.  Efficient Nonparametric Bayesian Modelling with Sparse Gaussian Process Approximations , 2006 .

[25]  A. Stuart,et al.  Sampling the posterior: An approach to non-Gaussian data assimilation , 2007 .

[26]  C.,et al.  Analysis methods for numerical weather prediction , 2022 .