Bayesian Calibration of Generalized Pools of Predictive Distributions

Decision-makers often consult different experts to build reliable forecasts on variables of interest. Combining more opinions and calibrating them to maximize the forecast accuracy is consequently a crucial issue in several economic problems. This paper applies a Bayesian beta mixture model to derive a combined and calibrated density function using random calibration functionals and random combination weights. In particular, it compares the application of linear, harmonic and logarithmic pooling in the Bayesian combination approach. The three combination schemes, i.e ., linear, harmonic and logarithmic, are studied in simulation examples with multimodal densities and an empirical application with a large database of stock data. All of the experiments show that in a beta mixture calibration framework, the three combination schemes are substantially equivalent, achieving calibration, and no clear preference for one of them appears. The financial application shows that the linear pooling together with beta mixture calibration achieves the best results in terms of calibrated forecast.

[1]  Robert Laddaga,et al.  Lehrer and the consensus proposal , 1977, Synthese.

[2]  M. Rosenblatt.  Remarks on a Multivariate Transformation , 1952 .

[3]  Mark J. Jensen,et al.  Bayesian Semiparametric Stochastic Volatility Modeling , 2008 .

[4]  Hemant Ishwaran,et al.  SERIES REPRESENTATIONS FOR MULTIVARIATE GENERALIZED GAMMA PROCESSES VIA A SCALE INVARIANCE PRINCIPLE , 2009 .

[5]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[6]  S. Hall,et al.  Combining density forecasts , 2007 .

[7]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[8]  Shaun P. Vahey,et al.  Combining forecast densities from VARs with uncertain instabilities , 2010 .

[9]  Norman R. Swanson,et al.  Predictive Density and Conditional Confidence Interval Accuracy Tests , 2004 .

[10]  Roberto Casarin,et al.  Bayesian Nonparametric Calibration and Combination of Predictive Distributions , 2015, 1502.07246.

[11]  C. Robert,et al.  A Mixture Approach to Bayesian Goodness of Fit , 2002 .

[12]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[13]  Matthew S. Johnson,et al.  Probabilistic wind gust forecasting using nonhomogeneous Gaussian regression , 2012 .

[14]  M. Stone.  The Opinion Pool , 1961 .

[15]  Kenneth F. Wallis,et al.  Density Forecasting: A Survey , 2000 .

[16]  Keisuke Hirano,et al.  Semiparametric Bayesian Inference in Autoregressive Panel Data Models , 2002 .

[17]  E. S. Epstein.  QUALITY CONTROL FOR PROBABILITY FORECASTS , 1966 .

[18]  T. Gneiting,et al.  Combining Predictive Distributions , 2011, 1106.1638.

[19]  Herman K. van Dijk,et al.  Parallel Sequential Monte Carlo for Efficient Density Combination: The Deco Matlab Toolbox , 2013 .

[20]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[21]  Jim E. Griffin,et al.  Inference in Infinite Superpositions of Non-Gaussian Ornstein--Uhlenbeck Processes Using Bayesian Nonparametic Methods , 2011 .

[22]  George Kapetanios,et al.  Generalised Density Forecast Combinations , 2014 .

[23]  A. V. D. Vaart,et al.  Posterior convergence rates of Dirichlet mixtures at smooth densities , 2007, 0708.1885.

[24]  Sonia Petrone.  Bayesian density estimation using bernstein polynomials , 1999 .

[25]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[26]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[27]  K. McConway.  Marginalization and Linear Opinion Pools , 1981 .

[28]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[29]  Nizar Bouguila,et al.  Practical Bayesian estimation of a finite beta mixture through gibbs sampling and its applications , 2006, Stat. Comput..

[30]  A. Timmermann.  Forecast Combinations , 2005 .

[31]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[32]  T. Gneiting,et al.  Combining probability forecasts , 2010 .

[33]  A. Timmermann.  Chapter 4 Forecast Combinations , 2006 .

[34]  M. Bacharach.  Group Decisions in the Face of Differences of Opinion , 1975 .

[35]  Jim E. Griffin,et al.  Stick-breaking autoregressive processes , 2011 .

[36]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[37]  A. Norets,et al.  Bayesian modeling of joint and conditional distributions , 2012 .

[38]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[39]  Roberto Casarin,et al.  Dynamic Predictive Density Combinations for Large Data Sets in Economics and Finance , 2015 .

[40]  H. V. Dijk,et al.  Combined forecasts from linear and nonlinear time series models , 1999 .

[41]  Massimo Guidolin,et al.  Forecasts of Us Short-Term Interest Rates: A Flexible Forecast Combination Approach , 2006 .

[42]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[43]  L. Wasserman,et al.  Consistency of Bernstein polynomial posteriors , 2002 .

[44]  M. Escobar.  Estimating Normal Means with a Dirichlet Process Prior , 1994 .

[45]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[46]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[47]  Carmen Cadarso-Suárez,et al.  Bayesian Nonparametric Instrumental Variables Regression Based on Penalized Splines and Dirichlet Process Mixtures , 2014 .

[48]  Vu,et al.  Time-Varying Combinations of Predictive Densities Using Nonlinear Filtering , 2012 .

[49]  Kenneth F. Wallis,et al.  Combining Density and Interval Forecasts: A Modest Proposal , 2005 .

[50]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[51]  Matt Taddy.  Autoregressive Mixture Models for Dynamic Spatial Poisson Processes: Application to Tracking Intensity of Violent Crime , 2010 .

[52]  Matt Taddy,et al.  Markov switching Dirichlet process mixture regression , 2009 .

[53]  G. A. Barnard,et al.  New Methods of Quality Control , 1963 .

[54]  M. A. Best.  Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[55]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[56]  John Geweke,et al.  Complete and Incomplete Econometric Models , 2010 .

[57]  H. Ishwaran,et al.  Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models , 2000 .

[58]  Justinas Pelenis.  Bayesian regression with heteroscedastic error density and parametric mean function , 2014 .

[59]  S. Ghosal,et al.  Kullback Leibler property of kernel mixture priors in Bayesian density estimation , 2007, 0710.2746.

[60]  John L. Kling,et al.  Calibration-Based Predictive Distributions: An Application of Prequential Analysis to Interest Rates, Money, Prices, and Output , 1989 .

[61]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[62]  S. Ghosal,et al.  Bayesian Estimation of the Spectral Density of a Time Series , 2004 .

[63]  Markus Jochmann,et al.  Modeling U.S. Inflation Dynamics: A Bayesian Nonparametric Approach , 2015 .

[64]  R. Casarin,et al.  Bayesian Model Selection for Beta Autoregressive Processes , 2010, 1008.0121.

[65]  Monica Billio,et al.  Beta Autoregressive Transition Markov-Switching Models for Business Cycle Analysis , 2011 .

[66]  T. M. Mills,et al.  Korovkin-type Approximation Theory and Its Applications, de Gruyter Studies in Mathematics 17, F. Altomare and M. Campiti, Walter de Gruyter, Berlin, 1994, xi + 627 pp , 1995 .

[67]  O. Papaspiliopoulos.  A note on posterior sampling from Dirichlet mixture models , 2008 .

[68]  Andriy Norets,et al.  POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES , 2011, Econometric Theory.

[69]  Tilmann Gneiting,et al.  Probabilistic forecasts of wind speed: ensemble model output statistics by using heteroscedastic censored regression , 2010 .

[70]  Pierre Pinson,et al.  Verification of the ECMWF ensemble forecasts of wind speed against analyses and observations , 2012 .

[71]  J. Geweke,et al.  Comparing and Evaluating Bayesian Predictive Distributions of Asset Returns , 2008 .

[72]  F. Bassetti,et al.  Beta-Product Dependent Pitman-Yor Processes for Bayesian Inference , 2013 .

[73]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[74]  J. Geweke,et al.  Optimal Prediction Pools , 2008 .

[75]  R. L. Winkler,et al.  Coherent combination of experts' opinions , 1995 .

[76]  Alexander Tsyplakov,et al.  Evaluating Density Forecasts: A Comment , 2011 .

[77]  S. Ghosal,et al.  Posterior consistency of Dirichlet mixtures for estimating a transition density , 2007 .

[78]  Yongqiang Tang,et al.  Nonparametric bayesian estimation of positive false discovery rates. , 2007, Biometrics.

[79]  Stephen G. Walker,et al.  Dependent mixtures of Dirichlet processes , 2011, Comput. Stat. Data Anal..

[80]  Germán Molina,et al.  A Bayesian Beta Markov Random Field Calibration of the Term Structure of Implied Risk Neutral Densities , 2014, 1409.1956.

[81]  C. Granger,et al.  Improved methods of combining forecasts , 1984 .

[82]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[83]  Sonia Petrone.  Random Bernstein Polynomials , 1999 .

[84]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[85]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[86]  Anthony S. Tay,et al.  Evaluating Density Forecasts with Applications to Financial Risk Management , 1998 .

[87]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[88]  D. Blei.  Bayesian Nonparametrics I , 2016 .

[89]  James Mitchell,et al.  Evaluating, Comparing and Combining Density Forecasts Using the Klic with an Application to the Bank of England and Niesr 'Fan' Charts of Inflation , 2005 .

[90]  James Mitchell,et al.  Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness , 2011 .

[91]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[92]  David B. Dunson,et al.  Posterior consistency in conditional distribution estimation , 2013, J. Multivar. Anal..

[93]  Thordis L. Thorarinsdottir,et al.  Comparison of nonhomogeneous regression models for probabilistic wind speed forecasting , 2018 .

[94]  M. Degroot,et al.  Optimal linear opinion pools , 1991 .

[95]  T. Gneiting,et al.  Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules , 2011 .

[96]  M. Harding,et al.  A Bayesian Semiparametric Competing Risk Model with Unobserved Heterogeneity , 2014 .

[97]  Enrique ter Horst,et al.  Bayesian dynamic density estimation , 2008 .

[98]  Bruno de Finetti,et al.  Logical foundations and measurement of subjective probability , 1970 .

[99]  L. Wasserman,et al.  Asymptotic inference for mixture models by using data‐dependent priors , 2000 .

[100]  Siddhartha Chib,et al.  Semiparametric Bayes analysis of longitudinal data treatment models , 2002 .

[101]  Stephen C. Hora,et al.  An Analytic Method for Evaluating the Performance of Aggregation Rules for Probability Densities , 2010, Oper. Res..