Object-Based Evaluation of a Storm-Scale Ensemble during the 2009 NOAA Hazardous Weather Testbed Spring Experiment

AbstractObject-based verification of deterministic forecasts from a convection-allowing ensemble for the 2009 NOAA Hazardous Weather Testbed Spring Experiment is conducted. The average of object attributes is compared between forecasts and observations and between forecasts from subensembles with different model dynamics. Forecast accuracy for the full ensemble and the subensembles with different model dynamics is also evaluated using two object-based measures: the object-based threat score (OTS) and the median of maximum interest (MMI).Forecast objects aggregated from the full ensemble are generally more numerous, have a smaller average area, more circular average aspect ratio, and more eastward average centroid location than observed objects after the 1-h lead time. At the 1-h lead time, forecast objects are less numerous than observed objects. Members using the Advanced Research Weather Research and Forecasting Model (ARW) have fewer objects, more linear average aspect ratio, and smaller average area t...

[1]  J. McBride,et al.  Verification of precipitation in weather systems: determination of systematic errors , 2000 .

[2]  B. Brown,et al.  Object-Based Verification of Precipitation Forecasts. Part I: Methodology and Application to Mesoscale Rain Areas , 2006 .

[3]  Jidong Gao,et al.  The Advanced Regional Prediction System (ARPS), storm-scale numerical weather prediction and data assimilation , 2003 .

[4]  W. Gallus.  Application of Object-Based Verification Techniques to Ensemble Precipitation Forecasts , 2010 .

[5]  Adam J. Clark,et al.  Contributions of Mixed Physics versus Perturbed Initial/Lateral Boundary Conditions to Ensemble-Based Precipitation Forecast Skill , 2008 .

[6]  Gregor Skok,et al.  Object-Based Analysis and Verification of WRF Model Precipitation in the Low- and Midlatitude Pacific Ocean , 2010 .

[7]  F. Kong.  A real-time storm-scale ensemble forecast system: 2009 Spring Experiment , 2009 .

[8]  S. J. Weiss,et al.  Next-Day Convection-Allowing WRF Model Guidance: A Second Look at 2-km versus 4-km Grid Spacing , 2009 .

[9]  J. Done,et al.  The next generation of NWP: explicit forecasts of convection using the weather research and forecasting (WRF) model , 2004 .

[10]  G. Grell,et al.  Mesoscale Weather Prediction with the RUC Hybrid Isentropic–Terrain-Following Coordinate Model , 2004 .

[11]  Jason E. Nachamkin,et al.  Evaluation of Heavy Precipitation Forecasts Using Composite-Based Methods: A Distributions-Oriented Approach , 2005 .

[12]  Caren Marzban,et al.  Cluster Analysis for Verification of Precipitation Fields , 2006 .

[13]  Song‐You Hong,et al.  Improvement of the K-profile Model for the Planetary Boundary Layer based on Large Eddy Simulation Data , 2003 .

[14]  I. Jolliffe.  Uncertainty and Inference for Verification Measures , 2007 .

[15]  Fanyou Kong,et al.  Hierarchical Cluster Analysis of a Convection-Allowing Ensemble during the Hazardous Weather Testbed 2009 Spring Experiment. Part II: Ensemble Clustering over the Whole Experiment Period , 2011 .

[16]  B. Ferrier,et al.  A Double-Moment Multiple-Phase Four-Class Bulk Ice Scheme. Part I: Description , 1994 .

[17]  Daniel S. Wilks,et al.  Statistical Methods in the Atmospheric Sciences: An Introduction , 1995 .

[18]  Adam J. Clark,et al.  Growth of Spread in Convection-Allowing and Convection-Parameterizing Ensembles , 2010 .

[19]  F. Kong.  Real-Time Storm-Scale Ensemble Forecast 2008 Spring Experiment , 2008 .

[20]  Fanyou Kong,et al.  Evaluation of CAPS multi-model storm-scale ensemble forecast for the NOAA HWT 2010 Spring Experiment , 2010 .

[21]  Eric Gilleland,et al.  Computationally Efficient Spatial Forecast Verification Using Baddeley’s Delta Image Metric , 2008 .

[22]  S. J. Weiss,et al.  Assessing Advances in the Assimilation of Radar Data and Other Mesoscale Observations within a Collaborative Forecasting-Research Environment , 2010 .

[23]  H. D. Orville,et al.  Bulk Parameterization of the Snow Field in a Cloud Model , 1983 .

[24]  Paul J. Roebber,et al.  Synoptic regulation of the 3 May 1999 tornado outbreak , 2002 .

[25]  S. J. Weiss,et al.  Probabilistic Precipitation Forecast Skill as a Function of Ensemble Size and Spatial Scale in a Convection-Allowing Ensemble , 2011 .

[26]  William A. Gallus,et al.  Spring and Summer Severe Weather Reports over the Midwest as a Function of Convective Mode: A Preliminary Study , 2008 .

[27]  Z. Janjic.  The Step-Mountain Eta Coordinate Model: Further Developments of the Convection, Viscous Sublayer, and Turbulence Closure Schemes , 1994 .

[28]  W. Skamarock.  Evaluating Mesoscale NWP Models Using Kinetic Energy Spectra , 2004 .

[29]  M. Xue,et al.  Convection-Allowing and Convection-Parameterizing Ensemble Forecasts of a Mesoscale Convective Vortex and Associated Severe Weather Environment , 2010 .

[30]  David M. Schultz,et al.  FORECASTER'S FORUM Toward Improved Prediction: High-Resolution and Ensemble Modeling Systems in Operations , 2004 .

[31]  D. Deaven,et al.  The Regional Analysis System for the Operational “Early” Eta Model: Original 80-km Configuration and Recent Changes , 1995 .

[32]  J. Dudhia,et al.  A Revised Approach to Ice Microphysical Processes for the Bulk Parameterization of Clouds and Precipitation , 2004 .

[33]  Z. Janjic.  A nonhydrostatic model based on a new approach , 2002 .

[34]  G. Bryan,et al.  Sensitivity of a Simulated Squall Line to Horizontal Resolution and Parameterization of Microphysics , 2012 .

[35]  John Manobianco,et al.  An Objective Technique for Verifying Sea Breezes in High-Resolution Numerical Weather Prediction Models , 2004 .

[36]  Xuguang Wang,et al.  Hierarchical Cluster Analysis of a Convection-Allowing Ensemble during the Hazardous Weather Testbed 2009 Spring Experiment. Part I: Development of the Object-Oriented Cluster Analysis Method for Precipitation Fields , 2011 .

[37]  J. Hansen,et al.  A parameterization for the absorption of solar radiation in the earth's atmosphere , 1974 .

[38]  Xuguang Wang,et al.  Verification and Calibration of Neighborhood and Object-Based Probabilistic Precipitation Forecasts from a Multimodel Convection-Allowing Ensemble , 2012 .

[39]  G. Thompson,et al.  Explicit Forecasts of Winter Precipitation Using an Improved Bulk Microphysics Scheme. Part II: Implementation of a New Snow Parameterization , 2008 .

[40]  W. Gallus,et al.  Comparison of the Diurnal Precipitation Cycle in Convection-Resolving and Non-Convection-Resolving Mesoscale Models , 2007 .

[41]  Thomas M. Hamill,et al.  Hypothesis Tests for Evaluating Numerical Precipitation Forecasts , 1999 .

[42]  Steven E. Koch,et al.  The use of a modified Ebert-McBride technique to evaluate mesoscale model QPF as a function of convective system morphology during IHOP 2002 , 2006 .

[43]  Ming Xue,et al.  CAPS REALTIME 4-KM MULTI-MODEL CONVECTION-ALLOWING ENSEMBLE AND 1-KM CONVECTION-RESOLVING FORECASTS FOR THE NOAA HAZARDOUS WEATHER TESTBED 2009 SPRING EXPERIMENT , 2009 .

[44]  B. Brown,et al.  Object-Based Verification of Precipitation Forecasts. Part II: Application to Convective Rain Systems , 2006 .

[45]  Alexander Khain,et al.  Microphysics, Radiation and Surface Processes in the Goddard Cumulus Ensemble (GCE) Model , 2003 .

[46]  S. Weiss.  Examination of several different versions of the WRF model for the prediction of severe convective weather: The SPC/NSSL Spring Program 2004 , 2004 .

[47]  B. Hagemeyer.  A Lower-Tropospheric Thermodynamic Climatology for March through September: Some Implications for Thunderstorm Forecasting , 1991 .

[48]  Jian Zhang,et al.  Constructing Three-Dimensional Multiple-Radar Reflectivity Mosaics: Examples of Convective Storms and Stratiform Rain Echoes , 2005 .

[49]  Athanasios C. Micheas,et al.  Cell identification and verification of QPF ensembles using shape analysis techniques , 2007 .

[50]  Jonathan L. Case,et al.  Improving Numerical Weather Predictions of Summertime Precipitation over the Southeastern United States through a High-Resolution Initialization of the Surface State , 2011 .

[51]  William C. Skamarock,et al.  A time-split nonhydrostatic atmospheric model for weather research and forecasting applications , 2008, J. Comput. Phys..

[52]  Jidong Gao,et al.  A Three-Dimensional Variational Data Analysis Method with Recursive Filter for Doppler Radars , 2004 .

[53]  S. J. Weiss,et al.  Toward Improved Convection-Allowing Ensembles: Model Physics Sensitivities and Optimizing Probabilistic Guidance with Small Ensemble Membership , 2010 .

[54]  K. Droegemeier,et al.  The Advanced Regional Prediction System (ARPS) – A multi-scale nonhydrostatic atmospheric simulation and prediction model. Part I: Model dynamics and verification , 2000 .

[55]  Caren Marzban,et al.  Cluster Analysis for Object-Oriented Verification of Fields: A Variation , 2008 .

[56]  Elizabeth E. Ebert,et al.  Toward Better Understanding of the Contiguous Rain Area (CRA) Method for Spatial Forecast Verification , 2009 .

[57]  A. H. Murphy,et al.  The Coefficients of Correlation and Determination as Measures of performance in Forecast Verification , 1995 .

[58]  Steven J. Weiss,et al.  Evaluation of WRF Model Output for Severe Weather Forecasting from the 2008 NOAA Hazardous Weather Testbed Spring Experiment , 2010 .

[59]  Moti Segal,et al.  Improving probabilistic ensemble forecasts of convection through the application of QPF-POP relationships , 2011 .

[60]  John M. Wallace,et al.  Diurnal Variations in Precipitation and Thunderstorm Frequency over the Conterminous United States , 1975 .

[61]  B. Brown,et al.  The Method for Object-Based Diagnostic Evaluation (MODE) Applied to Numerical Forecasts from the 2005 NSSL/SPC Spring Program , 2009 .

[62]  Kevin W. Manning,et al.  Experiences with 0–36-h Explicit Convective Forecasts with the WRF-ARW Model , 2008 .

[63]  Eric Gilleland,et al.  Intercomparison of Spatial Forecast Verification Methods , 2009 .

[64]  G. Brier,et al.  Some applications of statistics to meteorology , 1958 .

[65]  J. Dudhia.  Numerical Study of Convection Observed during the Winter Monsoon Experiment Using a Mesoscale Two-Dimensional Model , 1989 .

[66]  Paul J. Roebber,et al.  Short-Range (0–48 h) Numerical Prediction of Convective Occurrence, Mode, and Location , 2003 .

[67]  M. Xue,et al.  A Comparison of Precipitation Forecast Skill between Small Convection-Allowing and Large Convection-Parameterizing Ensembles , 2009 .

[68]  M. Xue,et al.  3DVAR and Cloud Analysis with WSR-88D Level-II Data for the Prediction of the Fort Worth, Texas, Tornadic Thunderstorms. Part I: Cloud Analysis and Its Impact , 2006 .

[69]  C. Bishop,et al.  Improvement of ensemble reliability with a new dressing kernel , 2005 .

[70]  C. Frei,et al.  SAL—A Novel Quality Measure for the Verification of Quantitative Precipitation Forecasts , 2008 .

[71]  J. D. Tarpley,et al.  Implementation of Noah land surface model advances in the National Centers for Environmental Prediction operational mesoscale Eta model , 2003 .

[72]  M. Xue.  3B.1 CAPS REALTIME STORM-SCALE ENSEMBLE AND HIGH-RESOLUTION FORECASTS AS PART OF THE NOAA HAZARDOUS WEATHER TESTBED 2007 SPRING EXPERIMENT , 2007 .