BAMLSS: Bayesian Additive Models for Location, Scale, and Shape (and Beyond)

ABSTRACT Bayesian analysis provides a convenient setting for the estimation of complex generalized additive regression models (GAMs). Since computational power has tremendously increased in the past decade, it is now possible to tackle complicated inferential problems, for example, with Markov chain Monte Carlo simulation, on virtually any modern computer. This is one of the reasons why Bayesian methods have become increasingly popular, leading to a number of highly specialized and optimized estimation engines and with attention shifting from conditional mean models to probabilistic distributional models capturing location, scale, shape (and other aspects) of the response distribution. To embed many different approaches suggested in literature and software, a unified modeling architecture for distributional GAMs is established that exploits distributions, estimation techniques (posterior mode or posterior mean), and model terms (fixed, random, smooth, spatial,…). It is shown that within this framework implementing algorithms for complex regression problems, as well as the integration of already existing software, is relatively straightforward. The usefulness is emphasized with two complex and computationally demanding application case studies: a large daily precipitation climatology, as well as a Cox model for continuous time with space-time interactions. Supplementary material for this article is available online.

[1]  Daniel Müller,et al.  The anatomy of distributional preferences with group identity , 2019, Journal of Economic Behavior & Organization.

[2]  Michael Razen,et al.  Greed: Taking a deadly sin to the lab. , 2019, Journal of Behavioral and Experimental Economics.

[3]  M. Geiger,et al.  Correlation and coordination risk , 2019, Annals of Finance.

[4]  R. Sabalis,et al.  VIENNA, AUSTRIA: , 2019, Finding Edith.

[5]  S. Lang,et al.  Random scaling factors in Bayesian distributional regression models with an application to real estate data , 2019, Statistical Modelling.

[6]  Lionel Page,et al.  Guilt averse or reciprocal? Looking at behavioral motivations in the trust game , 2018, Journal of the Economic Science Association.

[7]  Lionel Page,et al.  Why did he do that? Using counterfactuals to study the effect of intentions in extensive form games , 2018 .

[8]  Achim Zeileis,et al.  On the Estimation of Standard Errors in Cognitive Diagnosis Models , 2018 .

[9]  Lionel Page,et al.  Can a Common Currency Foster a Shared Social Identity across Different Nations? The Case of the Euro , 2017 .

[10]  Florian Wickelmaier,et al.  Using recursive partitioning to account for parameter heterogeneity in multinomial processing tree models , 2017, Behavior Research Methods.

[11]  Michael Kirchler,et al.  Immaterial and monetary gifts in economic transactions: evidence from the field , 2017, Experimental Economics.

[12]  Jakob W. Messner,et al.  Simultaneous Ensemble Postprocessing for Multiple Lead Times with Standardized Anomalies , 2017 .

[13]  Achim Zeileis,et al.  Ensemble Post-Processing of Daily Precipitation Sums over Complex Terrain Using Censored High-Resolution Standardized Anomalies , 2017 .

[14]  M. Halla,et al.  The Intergenerational Causal Effect of Tax Evasion: Evidence from the Commuter Tax Allowance in Austria , 2017, Journal of the European Economic Association.

[15]  M. Kirchler,et al.  Cash Inflow and Trading Horizon in Asset Markets , 2017 .

[16]  Marcus A. Brubaker,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[17]  Simon N. Wood Just Another Gibbs Additive Modeler: Interfacing JAGS and mgcv , 2016 .

[18]  Nadja Klein,et al.  Scale-Dependent Priors for Variance Parameters in Structured Additive Distributional Regression , 2016 .

[19]  S. Lang,et al.  Estimation of Spatially Correlated Random Scaling Factors based on Markov Random Field Priors , 2016 .

[20]  Achim Zeileis,et al.  Spatio‐temporal precipitation climatology over complex terrain using a censored additive regression model , 2016, International journal of climatology : a journal of the Royal Meteorological Society.

[21]  Nikolaus Umlauf,et al.  Flexible Bayesian additive joint models with an application to type 1 diabetes research , 2016, Biometrical journal. Biometrische Zeitschrift.

[22]  James M. Walker,et al.  Provision of public goods: Unconditional and conditional donations from outsiders , 2016 .

[23]  Jakob W. Messner,et al.  Tricks for improving non-homogeneous regression for probabilistic precipitation forecasts: Perfect predictions, heavy tails, and link functions , 2016 .

[24]  E. Dutcher,et al.  Don't hate the player, hate the game: Uncovering the foundations of cheating in contests , 2016 .

[25]  Zhuang Fengqing,et al.  Patients’ Responsibilities in Medical Ethics , 2016 .

[26]  Matthias Schmid,et al.  Boosting joint models for longitudinal and time‐to‐event data , 2016, Biometrical journal. Biometrische Zeitschrift.

[27]  M. Halla,et al.  The Long-Lasting Shadow of the Allied Occupation of Austria on its Spatial Equilibrium , 2016, SSRN Electronic Journal.

[28]  Nadja Klein,et al.  Simultaneous inference in structured additive conditional copula regression models: a unifying Bayesian approach , 2016, Stat. Comput..

[29]  M. Walzl,et al.  Incentive Schemes, Private Information and the Double-Edged Role of Competition for Agents , 2016 .

[30]  Achim Zeileis,et al.  Predictive Bookmaker Consensus Model for the UEFA Euro 2016 , 2016 .

[31]  Achim Zeileis,et al.  A Toolkit for Stability Assessment of Tree-Based Learners , 2016 .

[32]  Achim Zeileis,et al.  Score-Based Tests of Differential Item Functioning in the Two-Parameter Model , 2016 .

[33]  Achim Zeileis,et al.  Why Does It Always Rain on Me? A Spatio-Temporal Analysis of Precipitation in Austria , 2016 .

[34]  S. Wood Just Another Gibbs Additive Modeller: Interfacing JAGS and mgcv , 2016, 1602.02539.

[35]  Thomas W. Yee,et al.  Vector Generalized Linear and Additive Models: With an Implementation in R , 2015 .

[36]  Nadja Klein,et al.  Bayesian structured additive distributional regression for multivariate responses , 2015 .

[37]  Nadja Klein,et al.  Bayesian structured additive distributional regression with an application to regional income inequality in Germany , 2015, 1509.05230.

[38]  Benjamin M. Taylor Spatial modelling of emergency service response times , 2015, 1503.07709.

[39]  Nadja Klein,et al.  Bayesian Generalized Additive Models for Location, Scale, and Shape for Zero-Inflated and Overdispersed Count Data , 2015 .

[40]  Thiago G. Martins,et al.  Penalising Model Component Complexity: A Principled, Practical Approach to Constructing Priors , 2014, 1403.4630.

[41]  L. Fahrmeir,et al.  Regression: Models, Methods and Applications , 2013 .

[42]  I. Auer,et al.  Trends in extreme temperature indices in Austria based on a new homogenised dataset , 2013 .

[43]  Lucinda McCann,et al.  Why Does It Always Rain on Me , 2011 .

[44]  James G. Scott,et al.  On the half-cauchy prior for a global scale parameter , 2011, 1104.4937.

[45]  A. Zeileis,et al.  Extended Model Formulas in R : Multiple Parts and Multiple Responses , 2010 .

[46]  Andrew Thomas,et al.  The BUGS project: Evolution, critique and future directions , 2009, Statistics in medicine.

[47]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[48]  Gareth O. Roberts,et al.  Examples of Adaptive MCMC , 2009 .

[49]  Stefan Lang,et al.  Simultaneous selection of variables and smoothing parameters in structured additive regression models , 2008, Comput. Stat. Data Anal..

[50]  Alan Y. Chiang,et al.  Generalized Additive Models: An Introduction With R , 2007, Technometrics.

[51]  Christopher J Paciorek,et al.  Bayesian Smoothing with Gaussian Processes Using Fourier Basis Functions in the spectralGP Package. , 2007, Journal of statistical software.

[52]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[53]  L. Fahrmeir,et al.  A Mixed Model Approach for Geoadditive Hazard Regression , 2007 .

[54]  P. Gustafson,et al.  Conservative prior distributions for variance parameters in hierarchical models , 2006 .

[55]  L. Fahrmeir,et al.  Geoadditive Survival Models , 2006 .

[56]  Andreas Brezger,et al.  Generalized structured additive regression based on Bayesian P-splines , 2006, Comput. Stat. Data Anal..

[57]  R. Rigby,et al.  Generalized additive models for location, scale and shape , 2005 .

[58]  S. Wood Stable and Efficient Multiple Smoothing Parameter Estimation for Generalized Additive Models , 2004 .

[59]  A. Gelman Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[60]  T. Kneib S01.1: Penalized structured additive regression for space-time data , 2004 .

[61]  Matt P. Wand,et al.  Smoothing and mixed models , 2003, Comput. Stat..

[62]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[63]  Radford M. Neal Slice Sampling , 2000, physics/0009028.

[64]  Peter K. Dunn,et al.  Randomized Quantile Residuals , 1996 .

[65]  R. Kohn,et al.  On Gibbs sampling for state space models , 1994 .

[66]  W. Barlow,et al.  Residuals for relative risk regression , 1988 .

[67]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[68]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[69]  C. E. Rogers,et al.  Symbolic Description of Factorial Models for Analysis of Variance , 1973 .

[70]  D. Krige A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[71]  Achim Zeileis,et al.  Non-homogeneous boosting for predictor selection in ensemble post-processing , 2017 .

[72]  Gottfried Tappeiner,et al.  Do methodical traps lead to wrong development strategies for welfare? A multilevel approach considering heterogeneity across industrialized and developing countries , 2016 .

[73]  A. Brezger,et al.  BayesX Software for Bayesian Inference in Structured Additive Regression Models Version 3 . 0 . 2 Methodology Manual Developed by , 2015 .

[74]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[75]  Benjamin Hofner,et al.  Variable Selection and Model Choice in Survival Models with Time-Varying Effects , 2008 .

[76]  L. Fahrmeir,et al.  PENALIZED STRUCTURED ADDITIVE REGRESSION FOR SPACE-TIME DATA: A BAYESIAN PERSPECTIVE , 2004 .

[77]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[78]  S. Walker Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[79]  Dani Gamerman,et al.  Sampling from the posterior distribution in generalized linear mixed models , 1997, Stat. Comput..

[80]  Gordon K. Smyth,et al.  Partitioned algorithms for maximum likelihood and other non-linear estimation , 1996, Stat. Comput..

[81]  T. Hastie,et al.  Statistical Models in S , 1991 .

[82]  For a list of recent papers see the backpages of this paper. Structured Additive Regression Models: An R Interface to BayesX , 2022 .