Bayesian non-parametric quantile process regression and estimation of marginal quantile effects.

Flexible estimation of multiple conditional quantiles is of interest in numerous applications, such as studying the effect of pregnancy-related factors on low and high birth weight. We propose a Bayesian non-parametric method to simultaneously estimate non-crossing, non-linear quantile curves. We expand the conditional distribution function of the response in I-spline basis functions where the covariate-dependent coefficients are modeled using neural networks. By leveraging the approximation power of splines and neural networks, our model can approximate any continuous quantile function. Compared to existing models, our model estimates all rather than a finite subset of quantiles, scales well to high dimensions, and accounts for estimation uncertainty. While the model is arbitrarily flexible, interpretable marginal quantile effects are estimated using accumulative local effect plots and variable importance measures. A simulation study shows that our model can better recover quantiles of the response distribution when the data is sparse, and an analysis of birth weight data is presented.

[1]  J. Daniel,et al.  Splines with Nonnegative B-Spline Coefficients , 1974 .

[2]  M. Fuentes,et al.  Journal of the American Statistical Association Bayesian Spatial Quantile Regression Bayesian Spatial Quantile Regression , 2022 .

[3]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[4]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[5]  Nan Chen,et al.  Modeling Regression Quantile Process Using Monotone B-Splines , 2017, Technometrics.

[6]  B. Reich,et al.  Bayesian Quantile Regression for Censored Data , 2013, Biometrics.

[7]  Emil Pitkin,et al.  Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation , 2013, 1309.6392.

[8]  Kasper Kristensen,et al.  No-U-turn sampling for fast Bayesian inference in ADMB and TMB: Introducing the adnuts and tmbstan R packages , 2018, PloS one.

[9]  A. Gelman,et al.  Rank-normalization, folding, and localization: An improved R-hat for assessing convergence Rank-Normalization, Folding, and Localization: An Improved (cid:2) R for Assessing Convergence of MCMC An assessing for assessing An improved (cid:2) R for assessing convergence of MCMC , 2020 .

[10]  M. Betancourt,et al.  Hamiltonian Monte Carlo for Hierarchical Models , 2013, 1312.0906.

[11]  Yun Yang,et al.  Joint Estimation of Quantile Planes Over Arbitrary Predictor Spaces , 2015, 1507.03130.

[12]  B. Reich Spatiotemporal quantile regression for detecting distributional changes in environmental processes , 2012, Journal of the Royal Statistical Society. Series C, Applied statistics.

[13]  J. Kadane,et al.  Simultaneous Linear Quantile Regression: A Semiparametric Bayesian Approach , 2012 .

[14]  John Salvatier,et al.  Probabilistic programming in Python using PyMC3 , 2016, PeerJ Comput. Sci..

[15]  Fábio Ferreira,et al.  Conditional Density Estimation with Neural Networks: Best Practices and Benchmarks , 2019, ArXiv.

[16]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[17]  R. Beatson,et al.  Restricted Range Approximation by Splines and Variational Inequalities , 1982 .

[18]  M. Fuentes,et al.  Multilevel quantile function modeling with application to birth outcomes , 2015, Biometrics.

[19]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[20]  Subhashis Ghosal,et al.  Bayesian non-parametric simultaneous quantile regression for complete and grid data , 2018, Comput. Stat. Data Anal..

[21]  Ann B. Lee,et al.  Nonparametric Conditional Density Estimation in a High-Dimensional Regression Setting , 2016, 1604.00540.

[22]  Aki Vehtari,et al.  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC , 2015, Statistics and Computing.

[23]  Christopher C. Stanley,et al.  Determinants of Low Birth Weight in Malawi: Bayesian Geo-Additive Modelling , 2015, PloS one.

[24]  Francisco C. Pereira,et al.  Beyond Expectation: Deep Joint Mean and Quantile Regression for Spatiotemporal Problems. , 2018, IEEE transactions on neural networks and learning systems.

[25]  Sumio Watanabe,et al.  A widely applicable Bayesian information criterion , 2012, J. Mach. Learn. Res..

[26]  J. Elsner,et al.  The increasing intensity of the strongest tropical cyclones , 2008, Nature.

[27]  Michael Betancourt,et al.  A Conceptual Introduction to Hamiltonian Monte Carlo , 2017, 1701.02434.

[28]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[29]  P. Gustafson,et al.  Conservative prior distributions for variance parameters in hierarchical models , 2006 .

[30]  V. Chernozhukov,et al.  QUANTILE AND PROBABILITY CURVES WITHOUT CROSSING , 2007, 0704.3649.

[31]  Radford M. Neal Bayesian Learning via Stochastic Dynamics , 1992, NIPS.

[32]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[33]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[34]  Rui Li,et al.  Deep Distribution Regression , 2019, Comput. Stat. Data Anal..

[35]  James O. Ramsay,et al.  Nonparametric density estimation for censored survival data: Regression‐spline approach , 1992 .

[36]  Jason Abrevaya The effects of demographics and maternal behavior on the distribution of birth outcomes , 2001 .

[37]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Alexander G. Gray,et al.  Fast Nonparametric Conditional Density Estimation , 2007, UAI.

[39]  Yufeng Liu,et al.  Stepwise multiple quantile regression estimation using non-crossing constraints , 2009 .

[40]  Daniel W. Apley,et al.  Visualizing the effects of predictor variables in black box supervised learning models , 2016, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[41]  Carlos Guestrin,et al.  Model-Agnostic Interpretability of Machine Learning , 2016, ArXiv.

[42]  Yufeng Liu,et al.  Simultaneous multiple non-crossing quantile regression estimation using kernel constraints , 2011, Journal of nonparametric statistics.

[43]  Sebastiano Calvo,et al.  Estimating growth charts via nonparametric quantile regression: a practical framework with application in ecology , 2012, Environmental and Ecological Statistics.

[44]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[45]  Soumya Ghosh,et al.  Model Selection in Bayesian Neural Networks via Horseshoe Priors , 2017, J. Mach. Learn. Res..

[46]  Alex J. Cannon,et al.  Non-crossing nonlinear regression quantiles by monotone composite quantile regression neural network, with application to rainfall extremes , 2018, Stochastic Environmental Research and Risk Assessment.

[47]  Xuming He Quantile Curves without Crossing , 1997 .

[48]  Yanan Fan,et al.  Regression Adjustment for Noncrossing Bayesian Quantile Regression , 2015, 1502.01115.

[49]  H. Bondell,et al.  Noncrossing quantile regression curve estimation. , 2010, Biometrika.

[50]  Brandon M. Greenwell,et al.  A Simple and Effective Model-Based Variable Importance Measure , 2018, ArXiv.