The Multiset Sampler

We introduce the multiset sampler (MSS), a new Metropolis–Hastings algorithm for drawing samples from a posterior distribution. The MSS is designed to be effective when the posterior has the feature that the parameters can be divided into two sets, X, the parameters of interest and Y, the nuisance parameters. We contemplate a sampler that iterates between X moves and Y moves. We consider the case where either (a) Y is discrete and lives on a finite set or (b) Y is continuous and lives on a bounded set. After presenting some background, we define a multiset and show how to construct a distribution on one. The construction may seem artificial and pointless at first, but several small examples illustrate its value. Finally, we demonstrate the MSS in several realistic examples and compare it with alternatives.

[1]  Yuguo Chen,et al.  Genetics and population analysis The evolutionary forest algorithm Scotland , 2007 .

[2]  Adrian F. M. Smith,et al.  A Bayesian CART algorithm , 1998 .

[3]  W H Wong,et al.  Dynamic weighting in Monte Carlo and optimization. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .

[5]  S. Kou,et al.  Equi-energy sampler with applications in statistical inference and statistical mechanics , 2005, math/0507080.

[6]  Michael A. West,et al.  Bayesian CART: Prior Specification and Posterior Simulation , 2007 .

[7]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[8]  Jun S. Liu,et al.  Multipoint metropolis method with application to hybrid Monte Carlo , 2001 .

[9]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[10]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[11]  B. Berg,et al.  Multicanonical algorithms for first order phase transitions , 1991 .

[12]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[13]  D. Landau,et al.  Efficient, multiple-range random walk algorithm to calculate the density of states. , 2000, Physical review letters.

[14]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[15]  Berend Smit,et al.  Understanding molecular simulation: from algorithms to applications , 1996 .

[16]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[17]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[18]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[19]  H. Chipman,et al.  Bayesian CART Model Search , 1998 .

[20]  F. Liang.  A Generalized Wang–Landau Algorithm for Monte Carlo Computation , 2005 .

[21]  W. Wong,et al.  Real-Parameter Evolutionary Monte Carlo With Applications to Bayesian Mixture Models , 2001 .

[22]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[23]  Wang,et al.  Nonuniversal critical dynamics in Monte Carlo simulations. , 1987, Physical review letters.

[24]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.