Nested Sampling Methods

Nested sampling (NS) computes parameter posterior distributions and makes Bayesian model comparison computationally feasible. Its strengths are the unsupervised navigation of complex, potentially multi-modal posteriors until a well-defined termination point. A systematic literature review of nested sampling algorithms and variants is presented. We focus on complete algorithms, including solutions to likelihood-restricted prior sampling. A new formulation of NS is presented, which casts the parameter space exploration as a search on a tree. Previously published ways of obtaining robust error estimates and dynamic variations of the number of live points are presented as special cases of this formulation.

[1]  F. Feroz,et al.  Multimodal nested sampling: an efficient and robust alternative to Markov Chain Monte Carlo methods for astronomical data analyses , 2007, 0704.3704.

[2]  Doreen Eichel,et al.  Data Analysis A Bayesian Tutorial , 2016 .

[3]  Will Handley,et al.  Compromise-free Bayesian neural networks , 2020, ArXiv.

[4]  C. Palmisano,et al.  The expression of the model uncertainty in measurements , 2014, 29th Conference on Precision Electromagnetic Measurements (CPEM 2014).

[5]  Farhan Feroz,et al.  BAMBI: blind accelerated multimodal Bayesian inference , 2011, 1110.2997.

[6]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[7]  F. Feroz,et al.  MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics , 2008, 0809.3437.

[8]  Lei Cao,et al.  Design-as-inference: Probability-based design of intermodal transportation networks , 2014 .

[9]  B. Nikolic.  Fitting and Comparison of Models of Radio Spectra , 2009, 0912.2317.

[10]  A. Merloni,et al.  X-ray spectral modelling of the AGN obscuring region in the CDFS: Bayesian model selection and catalogue , 2014, 1402.0004.

[11]  A. Vecchio,et al.  Bayesian coherent analysis of in-spiral gravitational wave signals with a detector network , 2009, 0911.3820.

[12]  Equidistribution testing with Bayes factors and the ECT , 2016 .

[13]  Martino Trassinelli,et al.  Bayesian data analysis tools for atomic physics , 2016, 1611.10189.

[14]  D. Wales,et al.  Nested basin-sampling. , 2019, Journal of chemical theory and computation.

[15]  Radford M. Neal.  Slice Sampling , 2003, The Annals of Statistics.

[16]  C. Robert,et al.  Contemplating Evidence: properties, extensions of, and alternatives to Nested Sampling , 2008 .

[17]  Nested sampling on non-trivial geometries , 2019, 1905.09110.

[18]  M. Trassinelli,et al.  Mean Shift Cluster Recognition Method Implementation in the Nested Sampling Algorithm , 2020, Entropy.

[19]  V. Turchin.  On the Computation of Multidimensional Integrals by the Monte-Carlo Method , 1971 .

[20]  B. Brewer,et al.  Properties of the Affine Invariant Ensemble Sampler in high dimensions , 2015, 1509.02230.

[21]  Gábor Csányi,et al.  Constant-pressure nested sampling with atomistic dynamics. , 2017, Physical review. E.

[22]  J. Ridder,et al.  DIAMONDS: A new Bayesian nested sampling tool - Application to peak bagging of solar-like oscillations , 2014, 1408.2515.

[23]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[24]  Gábor Csányi,et al.  Efficient sampling of atomic configurational spaces. , 2009, The journal of physical chemistry. B.

[25]  Clément Walter,et al.  Point process-based Monte Carlo estimation , 2014, Stat. Comput..

[26]  Gábor Csányi,et al.  Gaussian Processes: A Method for Automatic QSAR Modeling of ADME Properties , 2007, J. Chem. Inf. Model..

[27]  Joris De Ridder,et al.  DIAMONDS: a new Bayesian nested sampling tool , 2015, 1509.08311.

[28]  Florian Beutler,et al.  Ensemble slice sampling , 2020, Statistics and Computing.

[29]  E. Ford,et al.  Quantifying the Bayesian Evidence for a Planet in Radial Velocity Data , 2018, The Astronomical Journal.

[30]  Johannes Buchner,et al.  Collaborative Nested Sampling: Big Data versus Complex Physical Models , 2017, Publications of the Astronomical Society of the Pacific.

[31]  M. Habeck.  Nested sampling with demons , 2015 .

[32]  J. Skilling,et al.  Discussion of Nested Sampling for Bayesian Computations by John Skilling , 2007 .

[33]  M. Payne,et al.  Determining pressure-temperature phase diagrams of materials , 2015, 1503.03404.

[34]  M. Betancourt.  Nested Sampling with Constrained Hamiltonian Monte Carlo , 2010, 1005.0157.

[35]  W. Nowak,et al.  Model selection on solid ground: Rigorous comparison of nine ways to evaluate Bayesian model evidence , 2014, Water resources research.

[36]  Palminteri Stefano,et al.  Bayesian Model comparison. , 2016 .

[37]  J. Lépine,et al.  A GALAXY MODEL FROM TWO MICRON ALL SKY SURVEY STAR COUNTS IN THE WHOLE SKY, INCLUDING THE PLANE , 2013, 1308.6238.

[38]  M. Betancourt.  Cruising The Simplex: Hamiltonian Monte Carlo and the Dirichlet Distribution , 2010, 1010.3436.

[39]  M. P. Hobson,et al.  polychord: nested sampling for cosmology , 2015, Monthly Notices of the Royal Astronomical Society: Letters.

[40]  C. Robert,et al.  Properties of nested sampling , 2008, 0801.3887.

[41]  Ray A. Jarvis,et al.  Clustering Using a Similarity Measure Based on Shared Near Neighbors , 1973, IEEE Transactions on Computers.

[42]  Comparison of Numerical Methods for Evidence Calculation , 2007 .

[43]  Nicholas G. Polson,et al.  Split Sampling: Expectations, Normalisation and Rare Events , 2012, 1212.0534.

[44]  X. Zeng,et al.  Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods , 2016 .

[45]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[46]  Dirk P. Kroese,et al.  Unbiased and consistent nested sampling via sequential Monte Carlo , 2018, 1805.03924.

[47]  S. S. Wilks.  The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses , 1938 .

[48]  M. P. Hobson,et al.  Importance Nested Sampling and the MultiNest Algorithm , 2013, The Open Journal of Astrophysics.

[49]  M. Hobson,et al.  Efficient Bayesian inference for multimodal problems in cosmology , 2007, astro-ph/0701867.

[50]  Ning Xiang,et al.  Room acoustic modal analysis using Bayesian inference. , 2017, The Journal of the Acoustical Society of America.

[51]  Michael Betancourt,et al.  A Conceptual Introduction to Hamiltonian Monte Carlo , 2017, 1701.02434.

[52]  James G. Scott,et al.  Vertical-likelihood Monte Carlo , 2014, 1409.3601.

[53]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[54]  I. Hudson,et al.  New prior sampling methods for nested sampling - Development and testing , 2017 .

[55]  D. Parkinson,et al.  A Nested Sampling Algorithm for Cosmological Model Selection , 2005, astro-ph/0508461.

[56]  M. Pitkin,et al.  A nested sampling code for targeted searches for continuous gravitational waves from pulsars , 2017, 1705.08978.

[57]  Paul M. Goggans,et al.  Parallelized nested sampling , 2014 .

[58]  M. Hobson,et al.  nestcheck: diagnostic tests for nested sampling calculations , 2018, Monthly Notices of the Royal Astronomical Society.

[59]  David L Wild,et al.  Exploring the energy landscapes of protein folding simulations with Bayesian computation. , 2010, Biophysical journal.

[60]  Will Handley,et al.  Nested sampling with plateaus , 2021 .

[61]  S. Vempala.  Geometric Random Walks: a Survey , 2007 .

[62]  M. Trassinelli.  The Nested_fit Data Analysis Program , 2019, Proceedings.

[63]  J. Dunkley,et al.  Comparison of sampling techniques for Bayesian parameter estimation , 2013, 1308.2675.

[64]  Zoubin Ghahramani,et al.  Nested sampling for Potts models , 2005, NIPS.

[65]  Jonathan R Goodman,et al.  Ensemble samplers with affine invariance , 2010 .

[66]  A. Vecchio,et al.  Assigning confidence to inspiral gravitational wave candidates with Bayesian model selection , 2008, 0807.4483.

[67]  Brendon J. Brewer,et al.  Diffusive nested sampling , 2009, Stat. Comput..

[68]  Robert L. Smith,et al.  An analysis of a variation of hit-and-run for uniform sampling from general regions , 2011, TOMC.

[69]  J. Speagle.  dynesty: a dynamic nested sampling package for estimating Bayesian posteriors and evidences , 2019, Monthly Notices of the Royal Astronomical Society.

[70]  Edward Higson,et al.  Dynamic nested sampling: an improved algorithm for parameter estimation and evidence calculation , 2017, Statistics and Computing.

[71]  D. Frenkel,et al.  Superposition Enhanced Nested Sampling , 2014, Physical Review X.

[72]  A. Lasenby,et al.  polychord: next-generation nested sampling , 2015, 1506.00171.

[73]  Johannes Buchner,et al.  A statistical test for Nested Sampling algorithms , 2014, Statistics and Computing.

[74]  Wang,et al.  Replica Monte Carlo simulation of spin glasses. , 1986, Physical review letters.

[75]  Kevin H. Knuth,et al.  Entropy-Based Search Algorithm for Experimental Design , 2010, ArXiv.

[76]  Lei Cao,et al.  Combined-chain nested sampling for efficient Bayesian model comparison , 2017, Digit. Signal Process..

[77]  M. Hobson,et al.  Sampling Errors in Nested Sampling Parameter Estimation , 2017, Bayesian Analysis.

[78]  A. Pettitt,et al.  Recursive Pathways to Marginal Likelihood Estimation with Prior-Sensitivity Analysis , 2013, 1301.6450.

[79]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[80]  Brendon J. Brewer,et al.  Computing Entropies With Nested Sampling , 2017, Entropy.

[81]  J. Skilling.  Nested Sampling’s Convergence , 2009 .

[82]  Charles R. Keeton,et al.  On statistical uncertainty in nested sampling , 2011, 1102.0996.

[83]  Daniel Foreman-Mackey,et al.  DNest4: Diffusive Nested Sampling in C++ and Python , 2016, 1606.03757.

[84]  Michael Khanarian,et al.  Monte Carlo Methods Final Project Nests and Tootsie Pops : Bayesian Sampling with Monte Carlo , 2013 .

[85]  J. Skilling.  Bayesian computation in big spaces-nested sampling and Galilean Monte Carlo , 2012 .

[87]  B. Brewer.  Inference for Trans-dimensional Bayesian Models with Diffusive Nested Sampling , 2014, 1411.3921.

[88]  T. Moller,et al.  Modeling and Analysis Generic Interface for eXternal numerical codes (MAGIX) , 2012, 1210.6466.

[89]  Kevin R. Wheeler,et al.  A Model-Based Probabilistic Inversion Framework for Characterizing Wire Fault Detection Using TDR , 2011, IEEE Transactions on Instrumentation and Measurement.

[90]  Chemical and statistical models of the interstellar medium and star-forming regions , 2015 .

[91]  Steven O Nielsen,et al.  Nested sampling in the canonical ensemble: direct calculation of the partition function from NVT trajectories. , 2013, The Journal of chemical physics.

[92]  J. Skilling.  Nested sampling for general Bayesian computation , 2006 .

[93]  Will Handley,et al.  Nested sampling cross-checks using order statistics , 2020, 2006.03371.