An Application of the Laplace Method to Finite Mixture Distributions

Abstract An exact Bayesian analysis of finite mixture distributions is often computationally infeasible, because the number of terms in the posterior density grows exponentially with the sample size. A modification of the Laplace method is presented and applied to estimation of posterior functions in a Bayesian analysis of finite mixture distributions. The procedure, which involves computations similar to those required in maximum likelihood estimation, is shown to have high asymptotic accuracy for finite mixtures of certain exponential-family densities. For these mixture densities, the posterior density is also shown to be asymptotically normal. An approximation of the posterior density of the number of components is presented. The method is applied to Duncan's barley data and to a distribution of lake chemistry data for north-central Wisconsin.

[1]  A. Wald Note on the Consistency of the Maximum Likelihood Estimate , 1949 .

[2]  D. B. Duncan MULTIPLE RANGE AND MULTIPLE F TESTS , 1955 .

[3]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .

[4]  H. Teicher Identifiability of Mixtures , 1961 .

[5]  Bruce M. Hill,et al.  The Three-Parameter Lognormal Distribution and Bayesian Analysis of a Point-Source Epidemic , 1963 .

[6]  F. Mosteller,et al.  Inference and Disputed Authorship: The Federalist , 1966 .

[7]  S. Yakowitz,et al.  On the Identifiability of Finite Mixtures , 1968 .

[8]  S. Goldfeld,et al.  A Markov model for switching regressions , 1973 .

[9]  A. Scott,et al.  A Cluster Analysis Method for Grouping Means in the Analysis of Variance , 1974 .

[10]  D. M. Titterington,et al.  Updating a Diagnostic System using Unconfirmed Cases , 1976 .

[11]  D. Binder Bayesian cluster analysis , 1978 .

[12]  D. Lindley,et al.  Approximate Bayesian methods , 1980 .

[13]  R. Redner Note on the Consistency of the Maximum Likelihood Estimate for Nonidentifiable Distributions , 1981 .

[14]  Michael J. Symons,et al.  Clustering criteria and multivariate normal mixtures , 1981 .

[15]  David A. Binder,et al.  Approximations to Bayesian clustering rules , 1981 .

[16]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[17]  R. Hathaway A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions , 1985 .

[18]  Chan‐Fu Chen On Asymptotic Normality of Limiting Density Functions with Bayesian Implications , 1985 .

[19]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[20]  M. Small,et al.  A regional pH-alkalinity relationship , 1986 .

[21]  Jerald L. Schnoor,et al.  Lake resources at risk to acidic deposition in the Upper Midwest , 1986 .

[22]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[23]  M. Small,et al.  A direct distribution model for regional aquatic acidification , 1986 .

[24]  M J Small,et al.  Parametric distributions of regional lake chemistry: fitted and derived. , 1988, Environmental science & technology.

[25]  L. Tierney,et al.  Fully Exponential Laplace Approximations to Expectations and Variances of Nonpositive Functions , 1989 .

[26]  John S. J. Hsu,et al.  Bayesian Marginal Inference , 1989 .

[27]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[28]  L. Tierney,et al.  The validity of posterior expansions based on Laplace''s method , 1990 .

[29]  M. Degroot,et al.  Modeling lake-chemistry distributions: approximate Bayesian methods for estimating a finite-mixture model , 1992 .

[30]  M. West,et al.  A Bayesian method for classification and discrimination , 1992 .