Fully Nonparametric Regression for Bounded Data Using Dependent Bernstein Polynomials

ABSTRACT We propose a novel class of probability models for sets of predictor-dependent probability distributions with bounded domain. The proposal extends the Dirichlet–Bernstein prior for single density estimation, by using dependent stick-breaking processes. A general model class and two simplified versions are discussed in detail. Appealing theoretical properties such as continuity, association structure, marginal distribution, large support, and consistency of the posterior distribution are established for all models. The behavior of the models is illustrated using simulated and real-life data. The simulated data are also used to compare the proposed methodology to existing methods. Supplementary materials for this article are available online.

[1]  L. Wasserman,et al.  Consistency of Bernstein polynomial posteriors , 2002 .

[2]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[3]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[4]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[5]  David B. Dunson,et al.  Posterior consistency in conditional distribution estimation , 2013, J. Multivar. Anal..

[6]  J. Jackson.  Wiley Series in Probability and Mathematical Statistics , 2004 .

[7]  A. V. D. Vaart,et al.  Posterior convergence rates for Dirichlet mixtures of beta densities , 2008 .

[8]  Maria De Iorio,et al.  Bayesian semiparametric inference for multivariate doubly-interval-censored data , 2010, 1101.1415.

[9]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[10]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[11]  Arnaud Doucet,et al.  Bayesian Inference for Dynamic Models with Dirichlet Process Mixtures , 2006, 2006 9th International Conference on Information Fusion.

[12]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[13]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[14]  O. Barndorff-Nielsen.  On M-ancillarity , 1973 .

[15]  Edward Greenberg,et al.  A Predictive Approach to Model Selection and Multicollinearity , 1993 .

[16]  Fernando A. Quintana,et al.  On the Support of MacEachern’s Dependent Dirichlet Processes and Extensions , 2012 .

[17]  Matthew West,et al.  Bayesian factor regression models in the''large p , 2003 .

[18]  C. Robert,et al.  A Mixture Approach to Bayesian Goodness of Fit , 2002 .

[19]  D. Dunson,et al.  Kernel stick-breaking processes. , 2008, Biometrika.

[20]  S. MacEachern,et al.  An ANOVA Model for Dependent Random Measures , 2004 .

[21]  S. Walker.  Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[22]  T. Ferguson.  Prior Distributions on Spaces of Probability Measures , 1974 .

[23]  David B. Dunson,et al.  Logistic Stick-Breaking Process , 2011, J. Mach. Learn. Res..

[24]  A. Gelfand,et al.  Bayesian analysis of proportional hazards models built from monotone functions. , 1995, Biometrics.

[25]  S. Ghosal.  Convergence rates for density estimation with Bernstein polynomials , 2001 .

[26]  D. Dunson,et al.  Nonparametric Bayes Conditional Distribution Modeling With Variable Selection , 2009, Journal of the American Statistical Association.

[27]  Peter Müller,et al.  Semiparametric Bayesian classification with longitudinal markers , 2007, Journal of the Royal Statistical Society. Series C, Applied statistics.

[28]  N. Pillai,et al.  Bayesian density regression , 2007 .

[29]  A. Gelfand,et al.  The Nested Dirichlet Process , 2008 .

[30]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[31]  Sonia Petrone.  Random Bernstein Polynomials , 1999 .

[32]  Paul H. C. Eilers,et al.  Flexible smoothing with B-splines and penalties , 1996 .

[33]  P. Müller,et al.  Random Partition Models with Regression on Covariates. , 2010, Journal of statistical planning and inference.

[34]  Alejandro Jara,et al.  Applied Bayesian Non-and Semi-parametric Inference using DPpackage , 2007 .

[35]  O. Barndorff-Nielsen.  Information and Exponential Families in Statistical Theory , 1980 .

[36]  P. Müller,et al.  Bayesian curve fitting using multivariate normal mixtures , 1996 .

[37]  A. Lijoi,et al.  Bayesian inference with dependent normalized completely random measures , 2014, 1407.0482.

[38]  Brian J. Smith,et al.  boa: An R Package for MCMC Output Convergence Assessment and Posterior Inference , 2007 .

[39]  S. Lang,et al.  Bayesian P-Splines , 2004 .

[40]  Bani K. Mallick,et al.  Generalized linear models with unknown link functions , 1994 .

[41]  D. Dunson,et al.  Multiscale Bernstein polynomials for densities , 2014, 1410.0827.

[42]  Jayanta K. Ghosh,et al.  Bayesian density regression with logistic Gaussian process and subspace projection , 2010 .

[43]  D. Dunson,et al.  Semiparametric Bayesian latent trajectory models , 2006 .

[44]  D. Dunson,et al.  Sparse Bayesian infinite factor models. , 2011, Biometrika.

[45]  O. Barndorff-Nielsen.  Information And Exponential Families , 1970 .

[46]  S. MacEachern,et al.  Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing , 2005 .

[47]  Sonia Petrone,et al.  A Predictive Study of Dirichlet Process Mixture Models for Curve Fitting , 2014, Scandinavian journal of statistics, theory and applications.

[48]  Wagner Barreto-Souza,et al.  Improved estimators for a general class of beta regression models , 2008, Comput. Stat. Data Anal..

[49]  Fernando A. Quintana,et al.  Bayesian density estimation for compositional data using random Bernstein polynomials , 2015 .

[50]  T. Hanson,et al.  A class of mixtures of dependent tail-free processes. , 2011, Biometrika.

[51]  S. MacEachern,et al.  A nonparametric Bayesian model for inference in related longitudinal studies , 2005 .

[52]  Radford M. Neal.  Slice Sampling , 2003, The Annals of Statistics.

[53]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[54]  Ilenia Epifani,et al.  Nonparametric priors for vectors of survival functions , 2009 .

[55]  Peter Müller,et al.  A Product Partition Model With Regression on Covariates , 2011, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[56]  Jean-Pierre Florens,et al.  Elements of Bayesian Statistics , 1990 .

[57]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[58]  D. Dunson,et al.  The local Dirichlet process , 2011, Annals of the Institute of Statistical Mathematics.

[59]  L. Carin,et al.  The Matrix Stick-Breaking Process , 2008 .

[60]  Peter Müller,et al.  DPpackage: Bayesian Semi- and Nonparametric Modeling in R , 2011 .

[61]  Andriy Norets,et al.  POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES , 2011, Econometric Theory.

[62]  Grace L. Yang,et al.  On Bayes Procedures , 1990 .

[63]  Bernardo Nipoti.  Dependent Completely Random Measures and Statistical Applications , 2012 .

[64]  S. Ferrari,et al.  Beta Regression for Modelling Rates and Proportions , 2004 .

[65]  S. MacEachern,et al.  Bayesian Density Estimation and Inference Using Mixtures , 2007 .

[66]  Fabrizio Leisen,et al.  Vectors of two-parameter Poisson-Dirichlet processes , 2011, J. Multivar. Anal..

[67]  Lorenzo Trippa,et al.  The multivariate beta process and an extension of the Polya tree model. , 2011, Biometrika.

[68]  J. Ghosh,et al.  POSTERIOR CONSISTENCY OF DIRICHLET MIXTURES IN DENSITY ESTIMATION , 1999 .

[69]  Wesley O Johnson,et al.  Bayesian Nonparametric Nonproportional Hazards Survival Modeling , 2009, Biometrics.

[70]  A. Kottas.  Dirichlet Process Mixtures of Beta Distributions , with Applications to Density and Intensity Estimation , 2006 .

[71]  Sonia Petrone.  Bayesian density estimation using bernstein polynomials , 1999 .

[72]  David B Dunson,et al.  Nonparametric Bayesian models through probit stick-breaking processes. , 2011, Bayesian analysis.

[73]  L. Tardella,et al.  Approximating distributions of random functionals of Ferguson‐Dirichlet priors , 1998 .