Maximum likelihood computation for fitting semiparametric mixture models

Three general algorithms that use different strategies are proposed for computing the maximum likelihood estimate of a semiparametric mixture model. They seek to maximize the likelihood function by, respectively, alternating the parameters, profiling the likelihood and modifying the support set. All three algorithms make a direct use of the recently proposed fast and stable constrained Newton method for computing the nonparametric maximum likelihood of a mixing distribution and employ additionally an optimization algorithm for unconstrained problems. The performance of the algorithms is numerically investigated and compared for solving the Neyman-Scott problem, overcoming overdispersion in logistic regression models and fitting two-level mixed effects logistic regression models. Satisfactory results have been obtained.

[1]  Dankmar Böhning,et al.  The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components , 2003, Stat. Comput..

[2]  Karen H. Haskell,et al.  An algorithm for linear least squares problems with equality and nonnegativity constraints , 1981, Math. Program..

[3]  B. Lindsay.  Mixture models : theory, geometry, and applications , 1995 .

[4]  Yong Wang.  On fast computation of the non‐parametric maximum likelihood estimate of a mixing distribution , 2007 .

[5]  J. Neyman,et al.  Consistent Estimates Based on Partially Consistent Observations , 1948 .

[6]  Yong Wang,et al.  Dimension-reduced nonparametric maximum likelihood computation for interval-censored data , 2008, Comput. Stat. Data Anal..

[7]  J. Heckman,et al.  A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data , 1984 .

[8]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[9]  R. Jennrich,et al.  Acceleration of the EM Algorithm by using Quasi‐Newton Methods , 1997 .

[10]  Murray Aitkin,et al.  A general maximum likelihood analysis of overdispersion in generalized linear models , 1996, Stat. Comput..

[11]  Diane Lambert,et al.  Generalizing Logistic Regression by Nonparametric Mixing , 1989 .

[12]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[13]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[14]  R. Dersimonian.  Maximum Likelihood Estimation of a Mixing Distribution , 1986 .

[15]  Donglin Zeng,et al.  Maximum likelihood estimation in semiparametric regression models with censored data , 2007, Statistica Sinica.

[16]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[17]  B. Lindsay.  The Geometry of Mixture Likelihoods: A General Theory , 1983 .

[18]  A Tsodikov,et al.  Semiparametric models: a generalized self‐consistency approach , 2003, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[19]  M. Aitkin.  A General Maximum Likelihood Analysis of Variance Components in Generalized Linear Models , 1999, Biometrics.

[20]  N. Laird.  Nonparametric Maximum Likelihood Estimation of a Mixing Distribution , 1978 .

[21]  Bruce G. Lindsay,et al.  A review of semiparametric mixture models , 1995 .

[22]  D. Böhning.  A review of reliable maximum likelihood algorithms for semiparametric mixture models , 1995 .

[23]  A. Dax.  The smallest point of a polytope , 1990 .

[24]  A. W. van der Vaart,et al.  On Profile Likelihood , 2000 .

[25]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .