On Monte Carlo methods for estimating ratios of normalizing constants

Recently, estimating ratios of normalizing constants has played an important role in Bayesian computations. Applications of estimating ratios of normalizing constants arise in many aspects of Bayesian statistical inference. In this article, we present an overview and discuss the current Monte Carlo methods for estimating ratios of normalizing constants. Then we propose a new ratio importance sampling method and establish its theoretical framework. We find that the ratio importance sampling method can be better than the current methods, for example, the bridge sampling method (Meng and Wong) and the path sampling method (Gelman and Meng), in the sense of minimizing asymptotic relative mean-square errors of estimators. An example is given for illustrative purposes. Finally, we present two special applications and the general implementation issues for estimating ratios of normalizing constants.

[1]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[2]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Rate of Convergence of One- and Two-Step $M$-Estimators with Applications to Maximum Likelihood and Pitman Estimators , 1985 .

[4]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[5]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[6]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[7]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[8]  Adrian F. M. Smith,et al.  Bayesian Analysis of Constrained Parameter and Truncated Data Problems , 1991 .

[9]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[10]  B. Schmeiser,et al.  Performance of the Gibbs, Hit-and-Run, and Metropolis Samplers , 1993 .

[11]  Ming-Hui Chen.  Importance-Weighted Marginal Bayesian Posterior Density Estimation , 1994 .

[12]  J. Geweke.  Bayesian comparison of econometric models , 1994 .

[13]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[14]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[15]  Q. Shao,et al.  A general bahadur representation of M-estimators and its application to linear regression with nonstochastic designs , 1996 .

[16]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[17]  Xiao-Li Meng,et al.  Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling , 1998 .