Strong Consistency of the Maximum Likelihood Estimator for Finite Mixtures of Location–Scale Distributions When Penalty is Imposed on the Ratios of the Scale Parameters

Abstract.  In finite mixtures of location–scale distributions, if there is no constraint or penalty on the parameters, then the maximum likelihood estimator does not exist because the likelihood is unbounded. To avoid this problem, we consider a penalized likelihood, where the penalty is a function of the minimum of the ratios of the scale parameters and the sample size. It is shown that the penalized maximum likelihood estimator is strongly consistent. We also analyse the consistency of a penalized maximum likelihood estimator where the penalty is imposed on the scale parameters themselves.

[1]  A. Wald.  Note on the Consistency of the Maximum Likelihood Estimate , 1949 .

[2]  N. E. Day.  Estimating the components of a mixture of normal distributions , 1969 .

[3]  R. Redner.  Note on the Consistency of the Maximum Likelihood Estimate for Nonidentifiable Distributions , 1981 .

[4]  S. Geman,et al.  Nonparametric Maximum Likelihood Estimation by the Method of Sieves , 1982 .

[5]  R. Hathaway.  A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions , 1985 .

[6]  L. Wasserman,et al.  RATES OF CONVERGENCE FOR THE GAUSSIAN MIXTURE SIEVE , 2000 .

[7]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[8]  A. V. D. Vaart,et al.  Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities , 2001 .

[9]  Strong consistency of MLE for finite mixtures of location-scale distributions when the scale parameters are exponentially small (full version) , 2006, math/0605148.

[10]  J. Idier,et al.  Penalized Maximum Likelihood Estimator for Normal Mixtures , 2000 .

[11]  A. Takemura,et al.  Strong consistency of MLE for finite uniform mixtures when the scale parameters are exponentially small , 2004, math/0409197.

[12]  Akimichi Takemura,et al.  Strong consistency of the maximum likelihood estimator for finite mixtures of location-scale distributions when the scale parameters are exponentially small , 2006 .

[13]  P. Deb.  Finite Mixture Models , 2008 .