Intercomparison of Spatial Forecast Verification Methods

Abstract Advancements in weather forecast models and their enhanced resolution have led to substantially improved and more realistic-appearing forecasts for some variables. However, traditional verification scores often indicate poor performance because of the increased small-scale variability so that the true quality of the forecasts is not always characterized well. As a result, numerous new methods for verifying these forecasts have been proposed. These new methods can mostly be classified into two overall categories: filtering methods and displacement methods. The filtering methods can be further delineated into neighborhood and scale separation, and the displacement methods can be divided into features based and field deformation. Each method gives considerably more information than the traditional scores, but it is not clear which method(s) should be used for which purpose. A verification methods intercomparison project has been established in order to glean a better understanding of the proposed me...

[1]  S. Weygandt.  Scale sensitivities in model precipitation skill scores during IHOP , 2004 .

[2]  Efi Foufoula-Georgiou,et al.  Space‐time rainfall organization and its role in validating quantitative precipitation forecasts , 2000 .

[3]  S. Lakshmivarahan,et al.  4 . 6 DEVELOPMENT OF AN EVENTS-ORIENTED VERIFICATION SYSTEM USING DATA MINING AND IMAGE PROCESSING ALGORITHMS , 2001 .

[4]  F. Atger.  Verification of intense precipitation forecasts from single models and ensemble prediction systems , 2001 .

[5]  C. Frei,et al.  SAL—A Novel Quality Measure for the Verification of Quantitative Precipitation Forecasts , 2008 .

[6]  Steven E. Koch,et al.  The use of a modified Ebert-McBride technique to evaluate mesoscale model QPF as a function of convective system morphology during IHOP 2002 , 2006 .

[7]  Jason E. Nachamkin,et al.  Evaluation of Heavy Precipitation Forecasts Using Composite-Based Methods: A Distributions-Oriented Approach , 2005 .

[8]  Christopher Grassotti,et al.  Feature calibration and alignment to represent model forecast errors: Empirical regularization , 2003 .

[9]  Caren Marzban,et al.  Cluster Analysis for Verification of Precipitation Fields , 2006 .

[10]  J. McBride,et al.  Verification of precipitation in weather systems: determination of systematic errors , 2000 .

[11]  W. Briggs.  Statistical Methods in the Atmospheric Sciences , 2007 .

[12]  I. Zawadzki,et al.  Scale Dependence of the Predictability of Precipitation from Continental Radar Images. Part II: Probability Forecasts , 2004 .

[13]  D. Stephenson,et al.  A new intensity‐scale approach for the verification of spatial precipitation forecasts , 2004 .

[14]  Marion Mittermaier,et al.  Using an intensity‐scale technique to assess the added benefit of high‐resolution model precipitation forecasts , 2006 .

[15]  Efi Foufoula-Georgiou,et al.  Scale‐recursive estimation for multisensor Quantitative Precipitation Forecast verification: A preliminary assessment , 2002 .

[16]  Harold E. Brooks,et al.  Verification of Nowcasts from the WWRP Sydney 2000 Forecast Demonstration Project , 2004 .

[17]  Caren Marzban,et al.  Cluster Analysis for Object-Oriented Verification of Fields: A Variation , 2008 .

[18]  Jason E. Nachamkin.  Application of the Composite Method to the Spatial Forecast Verification Methods Intercomparison Dataset , 2009 .

[19]  Neil I. Fox,et al.  An Object-Oriented Multiscale Verification Scheme , 2010 .

[20]  F. Mesinger.  Bias Adjusted Precipitation Threat Scores , 2008 .

[21]  Caren Marzban,et al.  Verification with Variograms , 2009 .

[22]  Elizabeth E. Ebert,et al.  Neighborhood Verification: A Strategy for Rewarding Close Forecasts , 2009 .

[23]  Heini Wernli,et al.  Spatial Forecast Verification Methods Intercomparison Project: Application of the SAL Technique , 2009 .

[24]  Efi Foufoula-Georgiou,et al.  A new metric for comparing precipitation patterns with an application to ensemble forecasts , 2005 .

[25]  V. Ducrocq,et al.  Point and areal validation of forecast precipitation fields , 2006 .

[26]  Richard A. Levine,et al.  Wavelets and Field Forecast Verification , 1997 .

[27]  R. Ogden,et al.  Essential Wavelets for Statistical Applications and Data Analysis , 1996 .

[28]  I. Jolliffe,et al.  Forecast verification : a practitioner's guide in atmospheric science , 2011 .

[29]  S. Dickinson,et al.  A Study of Near-Surface Winds in Marine Cyclones Using Multiple Satellite Sensors , 1996 .

[30]  Athanasios C. Micheas,et al.  Cell identification and verification of QPF ensembles using shape analysis techniques , 2007 .

[31]  Jason J. Levit,et al.  Multiscale Statistical Properties of a High-Resolution Precipitation Forecast , 2001 .

[32]  Elizabeth E. Ebert,et al.  Toward Better Understanding of the Contiguous Rain Area (CRA) Method for Spatial Forecast Verification , 2009 .

[33]  J. Kain,et al.  Sensitivity of Several Performance Measures to Displacement Error, Bias, and Event Frequency , 2006 .

[34]  C. Keil,et al.  A Displacement-Based Error Measure Applied in a Regional Ensemble Forecasting System , 2007 .

[35]  Barbara G. Brown,et al.  Forecast verification: current status and future directions , 2008 .

[36]  J. Louis,et al.  Distortion Representation of Forecast Errors , 1995 .

[37]  Jason E. Nachamkin,et al.  Mesoscale Verification Using Meteorological Composites , 2004 .

[38]  Clark F. Olson,et al.  View-Based Recognition Using an Eigenspace Approximation to the Hausdorff Measure , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  U. Damrath,et al.  Probabilistic precipitation forecasts from a deterministic model: a pragmatic approach , 2005 .

[40]  C. Keil,et al.  A Displacement and Amplitude Score Employing an Optical Flow Technique , 2009 .

[41]  Eric Gilleland,et al.  Application of Spatial Verification Methods to Idealized and NWP-Gridded Precipitation Forecasts , 2009 .

[42]  Nigel Roberts,et al.  Intercomparison of Spatial Forecast Verification Methods: Identifying Skillful Spatial Scales Using the Fractions Skill Score , 2010 .

[43]  Elizabeth E. Ebert,et al.  Fuzzy verification of high‐resolution gridded forecasts: a review and proposed framework , 2008 .

[44]  Daniel S. Wilks,et al.  Statistical Methods in the Atmospheric Sciences: An Introduction , 1995 .

[45]  B. Casati,et al.  New Developments of the Intensity-Scale Technique within the Spatial Verification Methods Intercomparison Project , 2010 .

[46]  B. Brown,et al.  The Method for Object-Based Diagnostic Evaluation (MODE) Applied to Numerical Forecasts from the 2005 NSSL/SPC Spring Program , 2009 .

[47]  Zbynek Sokol,et al.  A radar-based verification of precipitation forecast for local convective storms , 2007 .

[48]  B. Brown,et al.  Object-Based Verification of Precipitation Forecasts. Part I: Methodology and Application to Mesoscale Rain Areas , 2006 .

[49]  N. Roberts,et al.  Scale-Selective Verification of Rainfall Accumulations from High-Resolution Forecasts of Convective Events , 2008 .

[50]  Clark F. Olson,et al.  Object Recognition Using Subspace Methods , 1996, ECCV.

[51]  Eric Gilleland,et al.  Computationally Efficient Spatial Forecast Verification Using Baddeley’s Delta Image Metric , 2008 .

[52]  V. M. Karyampudi,et al.  The Effect of Assimilating Rain Rates Derived from Satellites and Lightning on Forecasts of the 1993 Superstorm , 1999 .

[53]  Caren Marzban,et al.  Three Spatial Verification Techniques: Cluster Analysis, Variogram, and Optical Flow , 2009 .