Information gain as a score for probabilistic forecasts

A measure of the information added by a probabilistic forecast to that contained in the climatological distribution is presented in this paper. This measure, called information gain, is mathematically closely related to the traditional ignorance score, but is more intuitive. Its advantages over other scores for probabilistic forecasts are also shown. The information gain score is tested on ECMWF ensemble forecasts of 500 hPa geopotential and 850 hPa temperature. The trends observed are in good agreement with those seen in other verification measures applied to the same data. In particular, the information gain decays with increasing lead time and increases over the years, in agreement with the improvement of the model. Copyright © 2010 Royal Meteorological Society

[1]  Jeffrey L. Anderson.  A Method for Producing and Evaluating Probabilistic Forecasts from Ensemble Model Integrations , 1996 .

[2]  Barbara G. Brown,et al.  Forecast verification: current status and future directions , 2008 .

[3]  Thomas M. Hamill,et al.  Verification of Eta–RSM Short-Range Ensemble Forecasts , 1997 .

[4]  Thomas M. Hamill,et al.  Comparison of Ensemble-MOS Methods Using GFS Reforecasts , 2007 .

[5]  Leonard A. Smith,et al.  Scoring Probabilistic Forecasts: The Importance of Being Proper , 2007 .

[6]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[7]  Leonard A. Smith,et al.  Evaluating Probabilistic Forecasts Using Information Theory , 2002 .

[8]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[9]  A. Sterl,et al.  The ERA‐40 re‐analysis , 2005 .

[10]  Daniel S. Wilks,et al.  Comparison of ensemble‐MOS methods in the Lorenz '96 setting , 2006 .

[11]  Leonard A. Smith,et al.  From ensemble forecasts to predictive distribution functions , 2008 .

[12]  H. Hersbach.  Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems , 2000 .

[13]  Edward S. Epstein,et al.  A Scoring System for Probability Forecasts of Ranked Categories , 1969 .

[14]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[15]  A. H. Murphy.  A Note on the Ranked Probability Score , 1971 .

[16]  Timothy DelSole,et al.  Predictability and Information Theory. Part II: Imperfect Forecasts , 2005 .

[17]  Bodo Ahrens,et al.  Information-Based Skill Scores for Probabilistic Forecasts , 2008 .

[18]  Leonard A. Smith,et al.  Combining dynamical and statistical ensembles , 2003 .

[19]  Timothy DelSole,et al.  Predictability and Information Theory. Part I: Measures of Predictability , 2004 .

[20]  R. L. Winkler,et al.  Scoring Rules for Continuous Probability Distributions , 1976 .

[21]  W. Briggs.  Statistical Methods in the Atmospheric Sciences , 2007 .

[22]  Fazlollah M. Reza,et al.  Introduction to Information Theory , 2004, Lecture Notes in Electrical Engineering.

[23]  J. Swets.  The Relative Operating Characteristic in Psychology , 1973, Science.

[24]  Renate Hagedorn,et al.  Communicating the value of probabilistic forecasts with weather roulette , 2009 .