Object-based evaluation of a numerical weather prediction model’s performance through forecast storm characteristic analysis

AbstractTraditional pixel-versus-pixel forecast evaluation scores such as the critical success index (CSI) provide a simple way to compare the performances of different forecasts; however, they offer little information on how to improve a particular forecast. This paper strives to demonstrate what additional information an object-based forecast evaluation tool such as the Method for Object-Based Diagnostic Evaluation (MODE) can provide in terms of assessing numerical weather prediction models’ convective storm forecasts. Forecast storm attributes evaluated by MODE in this paper include storm size, intensity, orientation, aspect ratio, complexity, and number of storms. Three weeks of the High Resolution Rapid Refresh (HRRR) model’s precipitation forecasts during the summer of 2010 over the eastern two-thirds of the contiguous United States were evaluated as an example to demonstrate the methodology. It is found that the HRRR model was able to forecast convective storm characteristics rather well either as ...

[1]  Matthias Steiner,et al.  Statistical assessment of explicit model forecasts of convection using a new object-based approach [presentation] , 2008 .

[2]  Stanley G. Benjamin,et al.  Advances in the Consolidated Storm Prediction for Aviation (CoSPA) [presentation] , 2010 .

[3]  B. Brown,et al.  The Method for Object-Based Diagnostic Evaluation (MODE) Applied to Numerical Forecasts from the 2005 NSSL/SPC Spring Program , 2009 .

[4]  Matthias Steiner,et al.  Assessment of the High-Resolution Rapid Refresh Model’s Ability to Predict Mesoscale Convective Systems Using Object-Based Evaluation , 2015 .

[5]  Eric Gilleland,et al.  Intercomparison of Spatial Forecast Verification Methods , 2009 .

[6]  Gregor Skok,et al.  Object-Based Analysis and Verification of WRF Model Precipitation in the Low- and Midlatitude Pacific Ocean , 2010 .

[7]  Curtis R. Alexander.  High resolution rapid refresh (HRRR): Recent enhancements and evaluation during the 2010 convective season , 2010 .

[8]  Michelle Harrold,et al.  Beyond the Basics: Evaluating Model-Based Precipitation Forecasts Using Traditional, Spatial, and Object-Based Methods , 2014 .

[9]  D. Stephenson,et al.  A new intensity‐scale approach for the verification of spatial precipitation forecasts , 2004 .

[10]  Fanyou Kong,et al.  Object-Based Evaluation of the Impact of Horizontal Grid Spacing on Convection-Allowing Forecasts , 2013 .

[11]  Jonathan L. Case,et al.  Improving Numerical Weather Predictions of Summertime Precipitation over the Southeastern United States through a High-Resolution Initialization of the Surface State , 2011 .

[12]  J. McBride,et al.  Verification of precipitation in weather systems: determination of systematic errors , 2000 .

[13]  Stephen S. Weygandt,et al.  The High Resolution Rapid Refresh (HRRR): an hourly updated convection resolving model utilizing radar reflectivity assimilation from the RUC / RR , 2009 .

[14]  B. Brown,et al.  Object-Based Verification of Precipitation Forecasts. Part II: Application to Convective Rain Systems , 2006 .

[15]  Robert A. Clark,et al.  Vertically Integrated Liquid Water—A New Analysis Tool , 1972 .

[16]  Paul J. Roebber,et al.  Assessing the Predictability of Convection Initiation in the High Plains Using an Object-Based Approach , 2014 .

[17]  M. Dixon,et al.  TITAN: Thunderstorm Identification, Tracking, Analysis, and Nowcasting—A Radar-based Methodology , 1993 .

[18]  G. Powers,et al.  A Description of the Advanced Research WRF Version 3 , 2008 .

[19]  B. Brown,et al.  Object-Based Verification of Precipitation Forecasts. Part I: Methodology and Application to Mesoscale Rain Areas , 2006 .

[20]  N. Roberts,et al.  Scale-Selective Verification of Rainfall Accumulations from High-Resolution Forecasts of Convective Events , 2008 .

[21]  M. Steiner,et al.  Consolidated storm prediction for aviation (CoSPA) , 2008, 2008 Integrated Communications, Navigation and Surveillance Conference.

[22]  Adam J. Clark,et al.  Application of Object-Based Time-Domain Diagnostics for Tracking Precipitation Systems in Convection-Allowing Models , 2014 .

[23]  John D. Tuttle,et al.  Inferences of Predictability Associated with Warm Season Precipitation Episodes , 2001 .

[24]  Juanzhen Sun,et al.  Multiple-Radar Data Assimilation and Short-Range Quantitative Precipitation Forecasting of a Squall Line Observed during IHOP_2002 , 2007 .

[25]  Kevin W. Manning,et al.  Explicit Forecasts of Winter Precipitation Using an Improved Bulk Microphysics Scheme. Part I: Description and Sensitivity Analysis , 2004 .

[26]  W. Skamarock.  Evaluating Mesoscale NWP Models Using Kinetic Energy Spectra , 2004 .

[27]  Zhiyong Meng,et al.  Quality of the Target Area for Metrics with Different Nonlinearities in a Mesoscale Convective System , 2014 .

[28]  Jordan G. Powers,et al.  A Description of the Advanced Research WRF Version 2 , 2005 .

[29]  Xuguang Wang,et al.  Object-Based Evaluation of a Storm-Scale Ensemble during the 2009 NOAA Hazardous Weather Testbed Spring Experiment , 2012 .

[30]  Michael Romanowski,et al.  Integrated communications navigation and surveillance conference(ICNS) , 2010, ICNS 2010.

[31]  Elizabeth E. Ebert,et al.  Fuzzy verification of high‐resolution gridded forecasts: a review and proposed framework , 2008 .