DNest4: Diffusive Nested Sampling in C++ and Python

In probabilistic (Bayesian) inferences, we typically want to compute properties of the posterior distribution, describing knowledge of unknown quantities in the context of a particular dataset and the assumed prior information. The marginal likelihood, also known as the "evidence", is a key quantity in Bayesian model selection. The Diffusive Nested Sampling algorithm, a variant of Nested Sampling, is a powerful tool for generating posterior samples and estimating marginal likelihoods. It is effective at solving complex problems including many where the posterior distribution is multimodal or has strong dependencies between variables. DNest4 is an open source (MIT licensed), multi-threaded implementation of this algorithm in C++11, along with associated utilities including: i) RJObject, a class template for finite mixture models, (ii) A Python package allowing basic use without C++ coding, and iii) Experimental support for models implemented in Julia. In this paper we demonstrate DNest4 usage through examples including simple Bayesian data analysis, finite mixture models, and Approximate Bayesian Computation.

[1]  Daniel Foreman-Mackey,et al.  emcee: The MCMC Hammer , 2012, 1202.3665.

[2]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[3]  Richard M. Stallman,et al.  Using the GNU Compiler Collection , 2010 .

[4]  Radford M. Neal Annealed importance sampling , 1998, Stat. Comput..

[5]  Brendon J. Brewer,et al.  Modelling reverberation mapping data – II. Dynamical modelling of the Lick AGN Monitoring Project 2008 data set , 2013, 1311.6475.

[6]  U. Hansmann Parallel tempering algorithm for conformational studies of biological molecules , 1997, physics/9710041.

[7]  J. Skilling Nested sampling for general Bayesian computation , 2006 .

[8]  A. L. Watts,et al.  DISSECTING MAGNETAR VARIABILITY WITH BAYESIAN HIERARCHICAL MODELS , 2015, 1501.05251.

[9]  D. Lindley Kendall's Advanced Theory of Statistics, volume 2B, Bayesian Inference, 2nd edn , 2005 .

[10]  Gábor Csányi,et al.  Efficient sampling of atomic configurational spaces. , 2009, The journal of physical chemistry. B.

[11]  Christian P Robert,et al.  Lack of confidence in approximate Bayesian computation model choice , 2011, Proceedings of the National Academy of Sciences.

[12]  Brendon J. Brewer,et al.  Fast Bayesian inference for exoplanet discovery in radial velocity data , 2015 .

[13]  Brendon J. Brewer,et al.  Modelling reverberation mapping data – I. Improved geometric and dynamical models and comparison with cross-correlation results , 2014, 1407.2941.

[14]  Bjarne Stroustrup,et al.  The C++ Programming Language, 4th Edition , 2013 .

[15]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[16]  P. Green Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[17]  John Skilling,et al.  Data analysis : a Bayesian tutorial , 1996 .

[18]  Richard Dybowski,et al.  Single passage in mouse organs enhances the survival and spread of Salmonella enterica , 2015, Journal of The Royal Society Interface.

[19]  Alan Edelman,et al.  Julia: A Fast Dynamic Language for Technical Computing , 2012, ArXiv.

[20]  F. Feroz,et al.  MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics , 2008, 0809.3437.

[21]  B. Brewer,et al.  Properties of the Affine Invariant Ensemble Sampler in high dimensions , 2015, 1509.02230.

[22]  B. Brewer Inference for Trans-dimensional Bayesian Models with Diffusive Nested Sampling , 2014, 1411.3921.

[23]  A. Lasenby,et al.  polychord: next-generation nested sampling , 2015, 1506.00171.

[24]  Brendon J. Brewer,et al.  Diffusive nested sampling , 2009, Stat. Comput..

[25]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[26]  Radford M. Neal Slice Sampling , 2003, The Annals of Statistics.

[27]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[28]  Robert John Nicholas Baldock,et al.  Research data supporting "Determining pressure-temperature phase diagrams of materials" , 2016 .

[29]  Arnaud Doucet,et al.  An adaptive sequential Monte Carlo method for approximate Bayesian computation , 2011, Statistics and Computing.

[30]  Stefan Behnel,et al.  Cython: The Best of Both Worlds , 2011, Computing in Science & Engineering.

[31]  Ariel Caticha,et al.  Updating Probabilities with Data and Moments , 2007, ArXiv.