On the Convergence of Successive Substitution Sampling

Abstract The problem of finding marginal distributions of multidimensional random quantities has many applications in probability and statistics. Many of the solutions currently in use are very computationally intensive. For example, in a Bayesian inference problem with a hierarchical prior distribution, one is often driven to multidimensional numerical integration to obtain marginal posterior distributions of the model parameters of interest. Recently, however, a group of Monte Carlo integration techniques that fall under the general banner of successive substitution sampling (SSS) have proven to be powerful tools for obtaining approximate answers in a very wide variety of Bayesian modeling situations. Answers may also be obtained at low cost, both in terms of computer power and user sophistication. Important special cases of SSS include the “Gibbs sampler” described by Gelfand and Smith and the “IP algorithm” described by Tanner and Wong. The major problem plaguing users of SSS is the difficulty in asce...

[1]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[2]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[3]  Sterling K. Berberian,et al.  Introduction to Hilbert Space , 1977 .

[4]  I. M. Glazman,et al.  Theory of linear operators in Hilbert space , 1961 .

[5]  Richard W. Hamming,et al.  Numerical Methods for Scientists and Engineers , 1963 .

[6]  Richard W. Hamming,et al.  Numerical Methods for Scientists and Engineers , 1962 .

[7]  Iosif Ilitch Gikhman,et al.  Introduction to the theory of random processes , 1969 .

[8]  George Weiss,et al.  Integral Equation Methods , 1969 .

[9]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[10]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Richard W. Hamming,et al.  Numerical methods for scientists and engineers (2nd ed.) , 1986 .

[12]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[13]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[14]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .

[15]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[16]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .