Summarizing multiple aspects of model performance in a single diagram

A diagram has been devised that can provide a concise statistical summary of how well patterns match each other in terms of their correlation, their root-mean-square difference, and the ratio of their variances. Although the form of this diagram is general, it is especially useful in evaluating complex models, such as those used to study geophysical phenomena. Examples are given showing that the diagram can be used to summarize the relative merits of a collection of different models or to track changes in performance of a model as it is modified. Methods are suggested for indicating on these diagrams the statistical significance of apparent differences and the degree to which observational uncertainty and unforced internal variability limit the expected agreement between model-simulated and observed behaviors. The geometric relationship between the statistics plotted on the diagram also provides some guidance for devising skill scores that appropriately weight among the various measures of pattern correspondence.

[1]  A. H. Murphy,et al.  Skill Scores Based on the Mean Square Error and Their Relationships to the Correlation Coefficient , 1988 .

[2]  A. H. Murphy,et al.  Skill Scores and Correlation Coefficients in Model Verification , 1989 .

[3]  B. Barkstrom,et al.  Seasonal variation of cloud radiative forcing derived from the Earth Radiation Budget Experiment , 1990 .

[4]  W. Rossow,et al.  ISCCP Cloud Data Products , 1991 .

[5]  A. Munot,et al.  All-India monthly and seasonal rainfall series: 1871–1993 , 1994 .

[6]  I. Watterson,et al.  Non-Dimensional Measures of Climate Model Performance , 1996 .

[7]  Ian T. Jolliffe,et al.  Revised “LEPS” Scores for Assessing Climate Model Simulations and Long-Range Forecasts , 1996 .

[8]  Daniel S. Wilks,et al.  Resampling Hypothesis Tests for Autocorrelated Fields , 1997 .

[9]  P. Xie,et al.  Global Precipitation: A 17-Year Monthly Analysis Based on Gauge Observations, Satellite Estimates, and Numerical Model Outputs , 1997 .

[10]  Jouni Räisänen,et al.  Objective comparison of patterns of CO2 induced climate change in coupled GCM experiments , 1997 .

[11]  I. Watterson,et al.  A comparison of present and doubled CO2 climates and feedbacks simulated by three general circulation models , 1999 .

[12]  S. M. Marlais,et al.  An Overview of the Results of the Atmospheric Model Intercomparison Project (AMIP I) , 1999 .

[13]  Mark New,et al.  Surface air temperature and its changes over the past 150 years , 1999 .

[14]  G. Boer,et al.  CMIP1 evaluation and intercomparison of coupled climate models , 2001 .

[15]  S. J. Lambert,et al.  Second-order space-time climate difference statistics , 2001 .