The calculation of posterior distributions by data augmentation

Abstract The idea of data augmentation arises naturally in missing value problems, as exemplified by the standard ways of filling in missing cells in balanced two-way tables. Thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze. This device is used to great advantage by the EM algorithm (Dempster, Laird, and Rubin 1977) in solving maximum likelihood problems. In situations when the likelihood cannot be approximated closely by the normal likelihood, maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements. From the Bayesian point of view, one must now calculate the posterior distribution of parameters of interest. If data augmentation can be used in the calculation of the maximum likelihood estimate, then in the same cases one ought to be able to use it in the computation of the posterior distribution. It is the purpose of this article to explain how this can be done. The basic idea ...

[1]  L. A. Goodman.  Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[2]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[3]  L. B. Rall,et al.  Computational Solution of Nonlinear Operator Equations , 1969 .

[4]  Calyampudi R. Rao,et al.  Linear Statistical Inference and Its Applications. , 1975 .

[5]  Peter E. Rossi,et al.  Bayesian analysis of dichotomous quantal response models , 1984 .

[6]  D. Rubin.  Multiple imputation for nonresponse in surveys , 1989 .

[7]  J. Schwartz,et al.  Linear Operators. Part I: General Theory. , 1960 .

[8]  S. Haberman.  Analysis of qualitative data , 1978 .

[9]  P. Odell,et al.  A Numerical Procedure to Generate a Sample Covariance Matrix , 1966 .

[10]  L. A. Goodman.  The Analysis of Systems of Qualitative Variables When Some of the Variables Are Unobservable. Part I-A Modified Latent Structure Approach , 1974, American Journal of Sociology.

[11]  G. C. Tiao,et al.  Bayesian estimation of latent roots and vectors with special reference to the bivariate normal distribution , 1969 .

[12]  J. E. H. Shaw,et al.  The implementation of the bayesian paradigm , 1985 .

[13]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[14]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[15]  J. Doob.  Stochastic processes , 1953 .

[16]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[17]  Kim-Hung Li,et al.  Imputation using Markov chains , 1988 .

[18]  D. Rubin.  Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician , 1984 .