Nonparametric estimation of convex models via mixtures

We present a general approach to estimating probability measures constrained to lie in a convex set. We represent constrained measures as mixtures of simple, known extreme measures, and so the problem of estimating a constrained measure becomes one of estimating an unconstrained mixing measure. Convex constraints arise in many modeling situations, such as estimation of the mean and estimation under stochastic ordering constraints. We describe mixture representation techniques for these and other situations, and discuss applications to maximum likelihood and Bayesian estimation.

[1]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[2]  D. L. Hanson,et al.  Maximum Likelihood Estimation of the Distributions of Two Stochastically Ordered Random Variables , 1966 .

[3]  K. Parthasarathy,et al.  Probability measures on metric spaces , 1967 .

[4]  S. Kullback.  Probability Densities with Given Marginals , 1968 .

[5]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[6]  R. M. Korwar,et al.  Contributions to the Theory of Dirichlet Processes , 1973 .

[7]  Extensions of measures. Stochastic equations , 1973 .

[8]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[9]  T. Ferguson.  Prior Distributions on Spaces of Probability Measures , 1974 .

[10]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[11]  R. Ash,et al.  Real analysis and probability , 1975 .

[12]  W. A. Coberly,et al.  The numerical evaluation of the maximum-likelihood estimate of mixture proportions , 1976 .

[13]  T. Kamae,et al.  Stochastic Inequalities on Partially Ordered Spaces , 1977 .

[14]  Changbao Wu,et al.  Some iterative procedures for generating nonsingular optimal designs , 1978 .

[15]  E. Dynkin.  Sufficient Statistics and Extreme Points , 1978 .

[16]  Gerhard Winkler,et al.  Integral representation in the set of solutions of a generalized moment problem , 1979 .

[17]  P. Pfanzagl,et al.  CONDITIONAL DISTRIBUTIONS AS DERIVATIVES , 1979 .

[18]  D. Freedman,et al.  Finite Exchangeable Sequences , 1980 .

[19]  G. Winkler,et al.  Non-Compact Extremal Integral Representations: Some Probabilistic Aspects , 1980 .

[20]  B. Lindsay.  The Geometry of Mixture Likelihoods: A General Theory , 1983 .

[21]  Albert Y. Lo,et al.  On a Class of Bayesian Nonparametric Estimates: I. Density Estimates , 1984 .

[22]  Hani Doss.  Bayesian Nonparametric Estimation of the Median; Part I: Computation of the Estimates , 1985 .

[23]  D. Freedman,et al.  On the consistency of Bayes estimates , 1986 .

[24]  A. Owen.  Empirical likelihood ratio confidence intervals for a single functional , 1988 .

[25]  J. Pfanzagl,et al.  Consistency of maximum likelihood estimators for certain nonparametric families, in particular: mixtures , 1988 .

[26]  Albert Y. Lo,et al.  Bayes Methods for a Symmetric Unimodal Density and its Mode , 1989 .

[27]  R. Dykstra,et al.  Nonparametric maximum likelihood estimation of survival functions with a general stochastic ordering and its dual , 1989 .

[28]  W. Chan,et al.  Unimodality, convexity, and applications , 1989 .

[29]  A. Owen.  Empirical Likelihood Ratio Confidence Regions , 1990 .

[30]  J. Kalbfleisch,et al.  An Algorithm for Computing the Nonparametric MLE of a Mixing Distribution , 1992 .

[31]  B. Leroux.  Consistent estimation of a mixing distribution , 1992 .

[32]  K. Roeder,et al.  Uniqueness of estimation and identifiability in mixture models , 1993 .

[33]  A. Karr,et al.  Point Processes and Their Statistical Inference. 2nd edn. , 1993 .

[34]  Hammou El Barmi,et al.  Restricted multinomial maximum likelihood estimation based upon Fenchel duality , 1994 .

[35]  D. Böhning.  A review of reliable maximum likelihood algorithms for semiparametric mixture models , 1995 .

[36]  B. Lindsay.  Mixture models : theory, geometry, and applications , 1995 .

[37]  M. Newton,et al.  Bayesian Inference for Semiparametric Binary Regression , 1996 .

[38]  Elja Arjas,et al.  Bayesian Inference of Survival Probabilities, under Stochastic Ordering Constraints , 1996 .

[39]  R. Theodorescu,et al.  Unimodality of Probability Measures , 1996 .

[40]  A. Raftery,et al.  A note on the Dirichlet process prior in Bayesian nonparametric inference with partial exchangeability , 1997 .

[41]  Sara van de Geer,et al.  Asymptotic Normality in Mixture Models , 1997 .

[42]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[43]  A. Forcina,et al.  A Unified Approach to Likelihood Inference on Stochastic Orderings in a Nonparametric Context , 1998 .

[44]  Hammou El Barmi,et al.  Maximum likelihood estimates via duality for log-convex models when cell probabilities are subject to convex constraints , 1998 .

[45]  Peter D. Hoff,et al.  Constrained Nonparametric Maximum Likelihood via Mixtures , 2000 .

[46]  Radford M. Neal.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[47]  Peter D. Hoff,et al.  Identifying Carriers of a Genetic Modifier Using Nonparametric Bayesian Methods , 2002 .