Spectral density-based and measure-preserving ABC for partially observed diffusion processes. An illustration on Hamiltonian SDEs

Approximate Bayesian computation (ABC) has become one of the major tools of likelihood-free statistical inference in complex mathematical models. Simultaneously, stochastic differential equations (SDEs) have developed to an established tool for modelling time-dependent, real-world phenomena with underlying random effects. When applying ABC to stochastic models, two major difficulties arise: First, the derivation of effective summary statistics and proper distances is particularly challenging, since simulations from the stochastic process under the same parameter configuration result in different trajectories. Second, exact simulation schemes to generate trajectories from the stochastic model are rarely available, requiring the derivation of suitable numerical methods for the synthetic data generation. To obtain summaries that are less sensitive to the intrinsic stochasticity of the model, we propose to build up the statistical method (e.g. the choice of the summary statistics) on the underlying structural properties of the model. Here, we focus on the existence of an invariant measure and we map the data to their estimated invariant density and invariant spectral density. Then, to ensure that these model properties are kept in the synthetic data generation, we adopt measure-preserving numerical splitting schemes. The derived property-based and measure-preserving ABC method is illustrated on the broad class of partially observed Hamiltonian type SDEs, both with simulated data and with real electroencephalography data. The derived summaries are particularly robust to the model simulation, and this fact, combined with the proposed reliable numerical scheme, yields accurate ABC inference. In contrast, the inference returned using standard numerical methods (Euler–Maruyama discretisation) fails. The proposed ingredients can be incorporated into any type of ABC algorithm and directly applied to all SDEs that are characterised by an invariant distribution and for which a measure-preserving numerical method can be derived.

[1]  Aaron Smith,et al.  The use of a single pseudo-sample in approximate Bayesian computation , 2014, Stat. Comput..

[2]  ABootstrapLikelihoodapproachtoBayesianComputation,et al.  A Bootstrap Likelihood Approach to Bayesian Computation , 2015, 1510.07287.

[3]  Christopher C. Drovandi,et al.  Pre-processing for approximate Bayesian computation in image analysis , 2015, Stat. Comput..

[4]  J. Møller Discussion on the paper by Feranhead and Prangle , 2012 .

[5]  Sergio Verdú,et al.  $f$ -Divergence Inequalities , 2015, IEEE Transactions on Information Theory.

[6]  Yanan Fan,et al.  Handbook of Approximate Bayesian Computation , 2018 .

[7]  Fernando Casas,et al.  Splitting and composition methods in the numerical integration of differential equations , 2008, 0812.0377.

[8]  S. Sisson,et al.  A comparative review of dimension reduction methods in approximate Bayesian computation , 2012, 1202.3819.

[9]  Paul Fearnhead,et al.  Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC , 2010, 1004.1112.

[10]  Jean-Marie Cornuet,et al.  Inferring population history with DIY ABC: a user-friendly approach to approximate Bayesian computation , 2008, Bioinform..

[11]  Dennis Prangle,et al.  Adapting the ABC distance function , 2015, 1507.00874.

[12]  Alexander Mendiburu,et al.  Distance Measures for Time Series in R: The TSdist Package , 2016, R J..

[13]  M. Gutmann,et al.  Fundamentals and Recent Developments in Approximate Bayesian Computation , 2016, Systematic biology.

[14]  Jean-Michel Marin,et al.  ABC random forests for Bayesian parameter inference , 2019, Bioinform..

[15]  Mathieu Gerber,et al.  Approximate Bayesian computation with the Wasserstein distance , 2019, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[16]  S. Wood Statistical inference for noisy nonlinear ecological dynamic systems , 2010, Nature.

[17]  Tetsuya Misawa,et al.  A Lie Algebraic Approach to Numerical Integration of Stochastic Differential Equations , 2001, SIAM J. Sci. Comput..

[18]  Samuel Soubeyrand,et al.  Approximate Bayesian computation with functional statistics , 2013, Statistical applications in genetics and molecular biology.

[19]  G. Quispel,et al.  Acta Numerica 2002: Splitting methods , 2002 .

[20]  Desmond J. Higham,et al.  Numerical simulation of a linear stochastic oscillator with additive noise , 2004 .

[21]  Dennis Prangle,et al.  Summary Statistics in Approximate Bayesian Computation , 2015, 1512.05633.

[22]  Stuart Barber,et al.  The Rate of Convergence for Approximate Bayesian Computation , 2013, 1311.2038.

[23]  I. Bojak,et al.  Fast approximate Bayesian inference for stable differential equation models , 2017, 1706.00689.

[24]  Fabrizio Leisen,et al.  A Bootstrap Likelihood Approach to Bayesian Computation , 2015, 1510.07287.

[25]  M. V. Tretyakov,et al.  Stochastic Numerics for Mathematical Physics , 2004, Scientific Computation.

[26]  Esteban Moro Egido,et al.  Boundary Preserving Semianalytic Numerical Algorithms for Stochastic Differential Equations , 2007, SIAM J. Sci. Comput..

[27]  G. Strang On the Construction and Comparison of Difference Schemes , 1968 .

[28]  A. J. Hermans,et al.  A model of the spatial-temporal characteristics of the alpha rhythm. , 1982, Bulletin of mathematical biology.

[29]  Umberto Picchini,et al.  Accelerating inference for diffusions observed with measurement error and large sample sizes using approximate Bayesian computation , 2013, 1310.0973.

[30]  Jonathan C. Mattingly,et al.  Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise , 2002 .

[31]  Kenneth N. Brown,et al.  Graph metrics as summary statistics for Approximate Bayesian Computation with application to network model parameter estimation , 2015, J. Complex Networks.

[32]  Christian P. Robert,et al.  Approximate Bayesian Computation: A Survey on Recent Results , 2014, MCQMC.

[33]  J. Bellanger,et al.  Epileptic fast activity can be explained by a model of impaired GABAergic dendritic inhibition , 2002, The European journal of neuroscience.

[34]  David Welch,et al.  Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems , 2009, Journal of The Royal Society Interface.

[35]  Michael P. H. Stumpf,et al.  Considerate approaches to constructing summary statistics for ABC model selection , 2012, Statistics and Computing.

[36]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[37]  Jean-Michel Marin,et al.  Approximate Bayesian computational methods , 2011, Statistics and Computing.

[38]  Raquel Prado,et al.  Bayesian mixture modeling for spectral density estimation , 2017 .

[39]  M. Muskulus,et al.  Wasserstein distances in the analysis of time series and dynamical systems , 2011 .

[40]  Bai Jiang,et al.  Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network , 2015, 1510.02175.

[41]  I. Vaughan L. Clarkson,et al.  On the periodogram estimators of periods from interleaved sparse, noisy timing data , 2014, 2014 IEEE Workshop on Statistical Signal Processing (SSP).

[42]  P. Donnelly,et al.  Inferring coalescence times from DNA sequence data. , 1997, Genetics.

[43]  Jeremy E. Oakley,et al.  Approximate Bayesian Computation and simulation based inference for complex stochastic epidemic models , 2018 .

[44]  B. Leimkuhler,et al.  Molecular Dynamics: With Deterministic and Stochastic Numerical Methods , 2015 .

[45]  Anthony N. Pettitt,et al.  Exact and Approximate Bayesian Inference for Low Integer-Valued Time Series Models with Intractable Likelihoods , 2016 .

[46]  Evelyn Buckwar,et al.  Splitting Integrators for the Stochastic Landau-Lifshitz Equation , 2016, SIAM J. Sci. Comput..

[47]  M. Blum Approximate Bayesian Computation: A Nonparametric Perspective , 2009, 0904.0635.

[48]  Adeline Samson,et al.  Coupling stochastic EM and approximate Bayesian computation for parameter inference in state-space models , 2015, Comput. Stat..

[49]  Brenda N. Vo,et al.  Quantifying uncertainty in parameter estimates for stochastic models of collective cell spreading using approximate Bayesian computation. , 2015, Mathematical biosciences.

[50]  Dirk Eddelbuettel,et al.  Rcpp: Seamless R and C++ Integration , 2011 .

[51]  Ajay Jasra,et al.  Approximate Bayesian Computation for a Class of Time Series Models , 2014, 1401.0265.

[52]  Darren J. Wilkinson,et al.  Bayesian inference for a discretely observed stochastic kinetic model , 2008, Stat. Comput..

[53]  S. Sisson,et al.  Diagnostic tools for approximate Bayesian computation using the coverage property , 2013, 1301.3166.

[54]  Libo Sun,et al.  Parameter inference and model selection in deterministic and stochastic dynamical models via approximate Bayesian computation: modeling a wildlife epidemic , 2014, 1409.7715.

[56]  Umberto Picchini Inference for SDE Models via Approximate Bayesian Computation , 2012, 1204.5459.

[57]  Charles-Edouard Br'ehier,et al.  Analysis of some splitting schemes for the stochastic Allen-Cahn equation , 2018, Discrete & Continuous Dynamical Systems - B.

[58]  Theodore Kypraios,et al.  A tutorial introduction to Bayesian inference for stochastic epidemic models using Approximate Bayesian Computation. , 2017, Mathematical biosciences.

[59]  G. Quispel,et al.  Splitting methods , 2002, Acta Numerica.

[60]  S. A. Sisson,et al.  ABC Samplers , 2018, Handbook of Approximate Bayesian Computation.

[61]  Ben H. Jansen,et al.  Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns , 1995, Biological Cybernetics.

[62]  Fabrice Wendling,et al.  Relevance of nonlinear lumped-parameter models in the analysis of depth-EEG epileptic signals , 2000, Biological Cybernetics.

[63]  Arnaud Guyader,et al.  New insights into Approximate Bayesian Computation , 2012, 1207.6461.

[64]  B. Leimkuhler,et al.  The computation of averages from equilibrium and nonequilibrium Langevin molecular dynamics , 2013, 1308.5814.

[65]  Jorge Caiado,et al.  A periodogram-based metric for time series classification , 2006, Comput. Stat. Data Anal..

[66]  David T. Frazier,et al.  Auxiliary Likelihood-Based Approximate Bayesian Computation in State Space Models , 2016, Journal of Computational and Graphical Statistics.

[67]  P. Kloeden,et al.  Numerical Solution of Stochastic Differential Equations , 1992 .

[68]  K Lehnertz,et al.  Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: dependence on recording region and brain state. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[69]  Andrea Tancredi,et al.  Approximate Bayesian inference for discretely observed continuous‐time multi‐state models , 2017, Biometrics.

[70]  A. Pettitt,et al.  Exact and approximate Bayesian inference for low count time series models with intractable likelihoods , 2016 .

[71]  R. Khasminskii Stability of Stochastic Differential Equations , 2012 .

[72]  D. Balding,et al.  Approximate Bayesian computation in population genetics. , 2002, Genetics.

[73]  Evelyn Buckwar,et al.  A Stochastic Version of the Jansen and Rit Neural Mass Model: Analysis and Numerics , 2017, The Journal of Mathematical Neuroscience.

[74]  Ben H. Jansen,et al.  A neurophysiologically-based mathematical model of flash visual evoked potentials , 2004, Biological Cybernetics.

[75]  D. Williams STOCHASTIC DIFFERENTIAL EQUATIONS: THEORY AND APPLICATIONS , 1976 .

[76]  Igor I. Gorban The Statistical Stability Phenomenon , 2016 .

[77]  Hui Jiang,et al.  Minimizing Sum of Truncated Convex Functions and Its Applications , 2016, Journal of Computational and Graphical Statistics.

[78]  Michael G. B. Blum,et al.  Choosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation , 2010, COMPSTAT.

[79]  Olivier François,et al.  Non-linear regression models for Approximate Bayesian Computation , 2008, Stat. Comput..

[80]  M. Feldman,et al.  Population growth of human Y chromosomes: a study of Y chromosome microsatellites. , 1999, Molecular biology and evolution.