An Algorithm for Computing the Nonparametric MLE of a Mixing Distribution

Abstract A fast algorithm for calculating the nonparametric maximum likelihood estimate (MLE) of a mixing distribution, G(·). in a mixture model is discussed. The literature contains several methods for computing the nonparametric MLE of G, but these methods are slow. In this article we develop an algorithm for maximizing the log-likelihood l(G) over the family ℊ, of all distribution functions, that yields the nonparametric MLE of G. In some semiparametric problems, a structural or fixed parameter β is of interest, and we are interested in computing profile likelihood, sup Gl(G, β), for a grid of values of β. The algorithm that we propose is fast enough for this purpose. Examples illustrate and compare the algorithms; one taken from an article by Laird, the common mean problem, and one taken from the literature on optimal experimental design. It is also noted that routines from the literature on semi-infinite programming may be used to compute the profile log-likelihood.

[1]  J. Neyman,et al.  Consistent Estimates Based on Partially Consistent Observations , 1948 .

[2]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .

[3]  J. Kiefer,et al.  The Equivalence of Two Extremum Problems , 1960, Canadian Journal of Mathematics.

[4]  H. Wynn.  The Sequential Generation of $D$-Optimum Experimental Designs , 1970 .

[5]  John D. Kalbfleisch,et al.  Application of Likelihood Methods to Models Involving Large Numbers of Parameters , 1970 .

[6]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[7]  P. Whittle.  Some General Points in the Theory of Optimal Experimental Design , 1973 .

[8]  L. Simar.  Maximum Likelihood Estimation of a Compound Poisson Process , 1976 .

[9]  Corwin L. Atwood,et al.  Convergent Design Sequences, for Sufficiently Regular Optimality Criteria , 1976 .

[10]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[11]  Changbao Wu,et al.  Some iterative procedures for generating nonsingular optimal designs , 1978 .

[12]  Changbao Wu,et al.  Some Algorithmic Aspects of the Theory of Optimal Designs , 1978 .

[13]  N. Laird.  Nonparametric Maximum Likelihood Estimation of a Mixing Distribution , 1978 .

[14]  Nicholas P. Jewell,et al.  Mixtures of Exponential Distributions , 1982 .

[15]  Numerical techniques for estimating probabilities , 1982 .

[16]  Dankmar Bohning.  Convergence of Simar's Algorithm for Finding the Maximum Likelihood Estimate of a Compound Poisson Process , 1982 .

[17]  B. Lindsay.  Conditional score functions: Some optimality results , 1982 .

[18]  B. Lindsay.  The Geometry of Mixture Likelihoods: A General Theory , 1983 .

[19]  Worst-case and average-case analysis of an algorithm solving a generalized knapsack problem , 1983 .

[20]  O. Barndorff-Nielsen.  On a formula for the distribution of the maximum likelihood estimator , 1983 .

[21]  A duality theorem with applications to statistics , 1983 .

[22]  J. Heckman,et al.  A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data , 1984 .

[23]  G. Alistair Watson,et al.  A projected lagrangian algorithm for semi-infinite programming , 1985, Math. Program..

[24]  Dankmar Böhning,et al.  Numerical estimation of a probability measure , 1985 .

[25]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[26]  Dankmar Böhning,et al.  A vertex-exchange-method in D-optimal design theory , 1986 .

[27]  D. Cox,et al.  Parameter Orthogonality and Approximate Conditional Inference , 1987 .

[28]  A. Gallant,et al.  Semi-nonparametric Maximum Likelihood Estimation , 1987 .

[29]  D. Böhning.  Likelihood inference for mixtures: Geometrical and other constructions of monotone step-length algorithms , 1989 .

[30]  Diane Lambert,et al.  Generalizing Logistic Regression by Nonparametric Mixing , 1989 .