Bayesian model averaging using particle filtering and Gaussian mixture modeling: Theory, concepts, and simulation experiments

Bayesian model averaging (BMA) is a standard method for combining predictive distributions from different models. In recent years, this method has enjoyed widespread application and use in many fields of study to improve the spread-skill relationship of forecast ensembles. The BMA predictive probability density function (pdf) of any quantity of interest is a weighted average of pdfs centered around the individual (possibly bias-corrected) forecasts, where the weights are equal to posterior probabilities of the models generating the forecasts, and reflect the individual models skill over a training (calibration) period. The original BMA approach presented by Raftery et al. (2005) assumes that the conditional pdf of each individual model is adequately described with a rather standard Gaussian or Gamma statistical distribution, possibly with a heteroscedastic variance. Here we analyze the advantages of using BMA with a flexible representation of the conditional pdf. A joint particle filtering and Gaussian mixture modeling framework is presented to derive analytically, as closely and consistently as possible, the evolving forecast density (conditional pdf) of each constituent ensemble member. The median forecasts and evolving conditional pdfs of the constituent models are subsequently combined using BMA to derive one overall predictive distribution. This paper introduces the theory and concepts of this new ensemble postprocessing method, and demonstrates its usefulness and applicability by numerical simulation of the rainfall-runoff transformation using discharge data from three different catchments in the contiguous United States. The revised BMA method receives significantly lower-prediction errors than the original default BMA method (due to filtering) with predictive uncertainty intervals that are substantially smaller but still statistically coherent (due to the use of a time-variant conditional pdf).

[1]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[2]  Martyn P. Clark,et al.  Ensemble Bayesian model averaging using Markov Chain Monte Carlo sampling , 2008 .

[3]  Roman Holenstein,et al.  Particle Markov chain Monte Carlo , 2009 .

[4]  T. Palmer,et al.  Development of a European Multi-Model Ensemble System for Seasonal to Inter-Annual Prediction (DEMETER) , 2004 .

[5]  Dirk P. Kroese,et al.  Global likelihood optimization via the cross-entropy method with an application to mixture models , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[6]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[7]  Lih-Yuan Deng,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning , 2006, Technometrics.

[8]  Jasper A. Vrugt,et al.  Hydrologic data assimilation using particle Markov chain Monte Carlo simulation: Theory, concepts and applications (online first) , 2012 .

[9]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[10]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[11]  Peter Jan,et al.  Particle Filtering in Geophysical Systems , 2009 .

[12]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting-II , 2005 .

[13]  H. Vereecken,et al.  Coupled hydrogeophysical parameter estimation using a sequential Bayesian approach , 2009 .

[14]  Cajo J. F. ter Braak,et al.  Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation , 2008 .

[15]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[16]  Andrew P. Morse,et al.  DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER) , 2004 .

[17]  Michael Ghil,et al.  Advanced data assimilation in strongly nonlinear dynamical systems , 1994 .

[18]  Roman Krzysztofowicz,et al.  A bivariate meta-Gaussian density for use in hydrology , 1997 .

[19]  T. N. Krishnamurti,et al.  Improved Weather and Seasonal Climate Forecasts from Multimodel Superensemble. , 1999, Science.

[20]  S. Sorooshian,et al.  Stochastic parameter estimation procedures for hydrologie rainfall‐runoff models: Correlated and heteroscedastic error cases , 1980 .

[21]  P. Moran.  Simulation and Evaluation of Complex Water Systems Operations , 1970 .

[22]  J. Vrugt,et al.  Corruption of accuracy and efficiency of Markov chain Monte Carlo simulation by inaccurate numerical implementation of conceptual hydrologic models , 2010 .

[23]  David S. Richardson,et al.  Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size , 2001 .

[24]  D. Higdon,et al.  Accelerating Markov Chain Monte Carlo Simulation by Differential Evolution with Self-Adaptive Randomized Subspace Sampling , 2009 .

[25]  A. Brath,et al.  A stochastic approach for assessing the uncertainty of rainfall‐runoff simulations , 2004 .

[26]  Jasper A. Vrugt,et al.  Comparison of point forecast accuracy of model averaging methods in hydrologic applications , 2010 .

[27]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — II. Calibration and combination , 2005 .

[28]  C. Diks,et al.  Improved treatment of uncertainty in hydrologic modeling: Combining the strengths of global optimization and data assimilation , 2005 .

[29]  A. Hense,et al.  A Bayesian approach to climate model evaluation and multi‐model averaging with an application to global mean surface temperatures from IPCC AR4 coupled climate models , 2006 .

[30]  Bruce A. Robinson,et al.  Treatment of uncertainty using ensemble methods: Comparison of sequential data assimilation and Bayesian model averaging , 2007 .

[31]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[32]  Kuolin Hsu,et al.  Uncertainty assessment of hydrologic model states and parameters: Sequential data assimilation using the particle filter , 2005 .

[33]  J. M. Sloughter,et al.  Probabilistic Wind Speed Forecasting Using Ensembles and Bayesian Model Averaging , 2010 .

[34]  E. Grimit,et al.  Initial Results of a Mesoscale Short-Range Ensemble Forecasting System over the Pacific Northwest , 2002 .

[35]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[36]  Thomas M. Hamill,et al.  Verification of Eta–RSM Short-Range Ensemble Forecasts , 1997 .

[37]  Dirk P. Kroese,et al.  The Cross Entropy Method: A Unified Approach To Combinatorial Optimization, Monte-carlo Simulation (Information Science and Statistics) , 2004 .

[38]  A. Barnston,et al.  Multimodel Ensembling in Seasonal Climate Forecasting at IRI , 2003 .

[39]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[40]  Balaji Rajagopalan,et al.  Categorical Climate Forecasts through Regularization and Optimal Combination of Multiple GCM Ensembles , 2002 .