A Bayesian Framework for Multimodel Regression

Abstract This paper presents a framework based on Bayesian regression and constrained least squares methods for incorporating prior beliefs in a linear regression problem. Prior beliefs are essential in regression theory when the number of predictors is not a small fraction of the sample size, a situation that leads to overfitting—that is, to fitting variability due to sampling errors. Under suitable assumptions, both the Bayesian estimate and the constrained least squares solution reduce to standard ridge regression. New generalizations of ridge regression based on priors relevant to multimodel combinations also are presented. In all cases, the strength of the prior is measured by a parameter called the ridge parameter. A “two-deep” cross-validation procedure is used to select the optimal ridge parameter and estimate the prediction error. The proposed regression estimates are tested on the Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER) hindcasts of s...

[1]  Francisco J. Doblas-Reyes,et al.  Forecast assimilation: a unified framework for the combination of multi-model weather and climate predictions , 2005 .

[2]  H. M. van den Dool,et al.  On the Weights for an Ensemble-Averaged 6–10-Day Forecast , 1994 .

[3]  T. N. Krishnamurti,et al.  Improvement of the Multimodel Superensemble Technique for Seasonal Forecasts , 2003 .

[4]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[5]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting-II , 2005 .

[6]  A. Barnston,et al.  Multimodel Ensembling in Seasonal Climate Forecasting at IRI , 2003 .

[7]  J. Michaelsen.  Cross-Validation in Statistical Climate Forecast Models , 1987 .

[8]  N. Draper,et al.  Applied Regression Analysis , 1967 .

[9]  C. M. Kishtawal,et al.  Multimodel Ensemble Forecasts for Weather and Seasonal Climate , 2000 .

[10]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[11]  A. E. Hoerl,et al.  Ridge Regression: Applications to Nonorthogonal Problems , 1970 .

[12]  Timothy DelSole,et al.  Predictability and Information Theory. Part II: Imperfect Forecasts , 2005 .

[13]  Norman R. Draper,et al.  Ridge Regression and James-Stein Estimation: Review and Comments , 1979 .

[14]  F. Zwiers,et al.  Climate Predictions with Multimodel Ensembles , 2002 .

[15]  Thomas M. Smith,et al.  Specification and Prediction of Global Surface Temperature and Precipitation from Global SST Using CCA , 1996 .

[16]  Upmanu Lall,et al.  Improved Combination of Multiple Atmospheric GCM Ensembles for Seasonal Prediction , 2004 .

[17]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[18]  Gary Smith,et al.  A Critique of Some Ridge Regression Methods , 1980 .

[19]  A. F. Smith,et al.  Ridge-Type Estimators for Regression Analysis , 1974 .

[20]  Andrew P. Morse,et al.  DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER) , 2004 .

[21]  J. Shukla,et al.  Specification of Wintertime North American Surface Temperature , 2006 .

[22]  P. Jones,et al.  Hemispheric and Large-Scale Surface Air Temperature Variations: An Extensive Revision and an Update to 2001. , 2003 .

[23]  Huug van den Dool,et al.  An analysis of multimodel ensemble predictions for seasonal climate anomalies , 2002 .

[24]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[25]  F. Joseph Turk,et al.  Real-time multianalysis-multimodel superensemble forecasts of precipitation using TRMM and SSM/I products , 2001 .

[26]  E. Jaynes.  Probability theory : the logic of science , 2003 .

[27]  T. N. Krishnamurti,et al.  Improved Weather and Seasonal Climate Forecasts from Multimodel Superensemble. , 1999, Science.

[28]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[29]  Balaji Rajagopalan,et al.  Categorical Climate Forecasts through Regularization and Optimal Combination of Multiple GCM Ensembles , 2002 .

[30]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[31]  R. Clemen.  Combining forecasts: A review and annotated bibliography , 1989 .