Using Artificial Intelligence to Improve Real-Time Decision-Making for High-Impact Weather

AbstractHigh-impact weather events, such as severe thunderstorms, tornadoes, and hurricanes, cause significant disruptions to infrastructure, property loss, and even fatalities. High-impact events can also positively impact society, such as the impact on savings through renewable energy. Prediction of these events has improved substantially with greater observational capabilities, increased computing power, and better model physics, but there is still significant room for improvement. Artificial intelligence (AI) and data science technologies, specifically machine learning and data mining, bridge the gap between numerical model prediction and real-time guidance by improving accuracy. AI techniques also extract otherwise unavailable information from forecast models by fusing model output with observations to provide additional decision support for forecasters and users. In this work, we demonstrate that applying AI techniques along with a physical understanding of the environment can significantly improve ...

[1]  T F Malone,et al.  APPLICATION OF STATISTICAL METHODS IN WEATHER PREDICTION. , 1955, Proceedings of the National Academy of Sciences of the United States of America.

[2]  B. M. Lewis,et al.  OBJECTIVE PREDICTION OF FIVE-DAY MEAN TEMPERATURES DURING WINTER , 1959 .

[3]  John T. Ball,et al.  The Diagnosis of Upper-Level Humidity , 1968 .

[4]  H. Glahn,et al.  The Use of Model Output Statistics (MOS) in Objective Weather Forecasting , 1972 .

[5]  C. Metz Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[6]  A. H. Murphy,et al.  The attributes diagram A geometrical framework for assessing the quality of probability forecasts , 1986 .

[7]  J. Key,et al.  Classification of merged AVHRR and SMMR Arctic data with neural networks , 1989 .

[8]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[9]  Richard L. Bankert,et al.  Cloud Classification of AVHRR Imagery in Maritime Regions Using a Probabilistic Neural Network , 1994 .

[10]  Wayne E. McGovern,et al.  The WSR-88D Severe Weather Potential Algorithm , 1995 .

[11]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[12]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[13]  Caren Marzban,et al.  A Neural Network for Tornado Prediction Based on Doppler Radar-Derived Attributes , 1996 .

[14]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[15]  Brian G. Smith,et al.  Use of regression techniques to predict hail size and the probability of large hail , 1997 .

[16]  A. Witt,et al.  An Enhanced Hail Detection Algorithm for the WSR-88D , 1998 .

[17]  Caren Marzban,et al.  A Neural Network for Damaging Wind Prediction , 1998 .

[18]  Kathleen L. Mosier,et al.  Does automation bias decision-making? , 1999, Int. J. Hum. Comput. Stud..

[19]  Lakshmanan,et al.  Identifying and tracking storms in satellite images , 2000 .

[20]  øöö Blockinøø Well-Trained PETs : Improving Probability Estimation , 2000 .

[21]  Caren Marzban,et al.  A Bayesian Neural Network for Severe-Hail Size Prediction , 2001 .

[22]  M. Richman,et al.  Euclidean Distance as a Similarity Metric for Principal Component Analysis , 2001 .

[23]  J. Friedman Stochastic gradient boosting , 2002 .

[24]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[25]  S. Franzblau,et al.  Use of Receiver Operating Characteristic Curves to Assess the Performance of a Microdilution Assay for Determination of Drug Susceptibility of Clinical Isolates of Mycobacterium tuberculosis , 2003, European Journal of Clinical Microbiology and Infectious Diseases.

[26]  Theodore B. Trafalis,et al.  Tornado Detection with Support Vector Machines , 2003, International Conference on Computational Science.

[27]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[28]  Arthur Witt,et al.  J 5 . 2 A Neural Network for Detecting and Diagnosing Tornadic Circulations using the Mesocyclone Detection and Near Storm Environment Algorithms , 2004 .

[29]  E. Anagnostou A convective/stratiform precipitation classification algorithm for volume scanning weather radar observations , 2004 .

[30]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[31]  Paul E. Johnson,et al.  Psychological Investigations of Competence in Decision Making , 2004 .

[32]  Barry E. Schwartz,et al.  An Hourly Assimilation–Forecast Cycle: The RUC , 2004 .

[33]  C. Moraga Introduction to Fuzzy Logic , 2005 .

[34]  Rich Caruana,et al.  Obtaining Calibrated Probabilities from Boosting , 2005, UAI.

[35]  S. Lakshmivarahan,et al.  Development of an Automated Classification Procedure for Rainfall Systems , 2005 .

[36]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[37]  George Tomlinson,et al.  Can Routine Laboratory Tests Discriminate between Severe Acute Respiratory Syndrome and Other Causes of Community-Acquired Pneumonia? , 2005, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[38]  Robert Sharman,et al.  An Integrated Approach to Mid- and Upper-Level Turbulence Forecasting , 2006 .

[39]  Sue Ellen Haupt,et al.  Source Characterization with a Genetic Algorithm–Coupled Dispersion–Backward Model Incorporating SCIPUFF , 2007 .

[40]  Travis M. Smith,et al.  An Automated Technique to Quality Control Radar Reflectivity Data , 2007 .

[41]  J. Facundo,et al.  METEOROLOGICAL ASSIMILATION DATA INGEST SYSTEM TRANSITION PROJECT RISK REDUCTION ACTIVITY , 2007 .

[42]  Sutherland,et al.  Statewide Monitoring of the Mesoscale Environment: A Technical Update on the Oklahoma Mesonet , 2007 .

[43]  Sue Ellen Haupt,et al.  Artificial Intelligence Methods in the Environmental Sciences , 2008 .

[44]  John K. Williams,et al.  Remote detection and diagnosis of thunderstorm turbulence , 2008, Optical Engineering + Applications.

[45]  John K. Williams,et al.  Combining observations and model data for short-term storm forecasting , 2008, Optical Engineering + Applications.

[46]  Adel Mellit,et al.  Artificial Intelligence technique for modelling and forecasting of solar radiation data: a review , 2008, Int. J. Artif. Intell. Soft Comput..

[47]  Amy McGovern,et al.  Classification of Convective Areas Using Decision Trees , 2009 .

[48]  Theodore B. Trafalis,et al.  Support vector machines for spatiotemporal tornado prediction , 2009, Int. J. Gen. Syst..

[49]  Stephen S. Weygandt,et al.  The High Resolution Rapid Refresh (HRRR): an hourly updated convection resolving model utilizing radar reflectivity assimilation from the RUC / RR , 2009 .

[50]  Katherine L. Milkman,et al.  How Can Decision Making Be Improved? , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[51]  Paul J. Roebber,et al.  Visualizing Multiple Measures of Forecast Quality , 2009 .

[52]  Valliappa Lakshmanan,et al.  A Technique to Censor Biological Echoes in Radar Reflectivity Data , 2010 .

[53]  M. Etezadi-Amoli,et al.  Practical approach for sub-hourly and hourly prediction of PV power output , 2010, North American Power Symposium 2010.

[54]  Prashant J. Shenoy,et al.  Predicting solar generation from weather forecasts using machine learning , 2011, 2011 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[55]  C. Coimbra,et al.  Forecasting of global and direct solar irradiance using stochastic learning methods, ground experiments and the NWS database , 2011 .

[56]  Sue Ellen Haupt,et al.  A consensus forecasting approach for improved turbine hub height wind speed predictions , 2011 .

[57]  N. Bashardoost,et al.  Kernel Smoothing For ROC Curve And Estimation For Thyroid Stimulating Hormone , 2011 .

[58]  Yubao Liu,et al.  Wind Energy Forecasting: A Collaboration of the National Center for Atmospheric Research (NCAR) and Xcel Energy , 2011 .

[59]  Kiel L. Ortega,et al.  The Multi-Year Reanalysis of Remotely Sensed Storms (MYRORSS) Project , 2012 .

[60]  P. Jirutitijaroen,et al.  Hourly solar irradiance time series forecasting using cloud cover index , 2012 .

[61]  Alexander V. Ryzhkov,et al.  Classification of precipitation types during transitional winter weather using the RUC model and polarimetric radar retrievals , 2012 .

[62]  Fei Wang,et al.  Short-Term Solar Irradiance Forecasting Model Based on Artificial Neural Network Using Statistical Feature Parameters , 2012 .

[63]  S. E. Haupt,et al.  A Wind Power Forecasting System to Optimize Grid Integration , 2012, IEEE Transactions on Sustainable Energy.

[64]  T. Fearn Ridge Regression , 2013 .

[65]  Michael B. Richman,et al.  Range-Correcting Azimuthal Shear in Doppler Radar Data , 2012 .

[66]  John K. Williams,et al.  Enhancing understanding and improving prediction of severe weather through spatiotemporal relational learning , 2013, Machine Learning.

[67]  Luca Delle Monache,et al.  Probabilistic Weather Prediction with an Analog Ensemble , 2013 .

[68]  Travis M. Smith,et al.  An Automated Method for Depicting Mesocyclone Paths and Intensities , 2013 .

[69]  Carlos F.M. Coimbra,et al.  Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning , 2013 .

[70]  Marc Keyser,et al.  Knowledge Is Power: Efficiently Integrating Wind Energy and Wind Forecasts , 2013, IEEE Power and Energy Magazine.

[71]  Agostino Manzato,et al.  Hail in Northeast Italy: A Neural Network Ensemble Forecast Using Sounding-Derived Indices , 2013 .

[72]  John K. Williams,et al.  Using random forests to diagnose aviation turbulence , 2013, Machine Learning.

[73]  Keith F. Brill,et al.  Precipitation and Temperature Forecast Performance at the Weather Prediction Center , 2014 .

[74]  M. Pavolonis,et al.  An Empirical Model for Assessing the Severe Weather Potential of Developing Convection , 2014 .

[75]  K. Elmore,et al.  Sources of Uncertainty in Precipitation-Type Forecasting , 2014 .

[76]  Valliappa Lakshmanan,et al.  Quality Control of Weather Radar Data Using Polarimetric Variables , 2014 .

[77]  Lans P. Rothfusz,et al.  MPING: Crowd-Sourcing Weather Reports for Research , 2014 .

[78]  John K. Williams,et al.  Probabilistic 0–1-h Convective Initiation Nowcasts that Combine Geostationary Satellite Observations and Numerical Weather Prediction Model Data , 2015 .

[79]  S. E. Haupt,et al.  A model tree approach to forecasting solar irradiance variability , 2015 .

[80]  Adam J. Clark,et al.  An Automated, Multiparameter Dryline Identification Algorithm , 2015 .

[81]  Amanda S. Hering,et al.  A Markov-Switching Vector Autoregressive Stochastic Wind Generator for Multiple Spatial and Temporal Scales , 2015 .

[82]  Darrel M. Kingfield,et al.  A Method for Extracting Postevent Storm Tracks , 2015 .

[83]  Bri-Mathias Hodge,et al.  Recent Trends in Variable Generation Forecasting and Its Value to the Power System , 2015, IEEE Transactions on Sustainable Energy.

[84]  K. Elmore,et al.  Verifying Forecast Precipitation Type with mPING , 2015 .

[85]  Kristin M. Calhoun,et al.  Evaluation of a Probabilistic Forecasting Methodology for Severe Convective Weather in the 2014 Hazardous Weather Testbed , 2015 .

[86]  L. D. Monache,et al.  An analog ensemble for short-term probabilistic solar power forecast , 2015 .

[87]  Sue Ellen Haupt,et al.  Solar Forecasting: Methods, Challenges, and Performance , 2015, IEEE Power and Energy Magazine.

[88]  A. McGovern,et al.  Solar Energy Prediction: An International Contest to Initiate Interdisciplinary Research on Compelling Meteorological Problems , 2015 .

[89]  Ryan Lagerquist,et al.  Using Machine Learning to Predict Damaging Straight-line Convective Winds , 2016 .

[90]  David John Gagne,et al.  Coupling Data Science Techniques and Numerical Weather Prediction Models for High-Impact Weather Prediction , 2016 .

[91]  John K. Williams,et al.  Remote Turbulence Detection Using Ground-Based Doppler Weather Radar , 2016 .

[92]  S. E. Haupt,et al.  Regime-Dependent Short-Range Solar Irradiance Forecasting , 2016 .

[93]  Rebecca D. Adams-Selin,et al.  Forecasting Hail Using a One-Dimensional Hail Growth Model within WRF , 2016 .

[94]  Matthias Steiner,et al.  Probabilistic forecasts of mesoscale convective system initiation using the random forest data mining technique , 2016 .

[95]  Craig S. Schwartz,et al.  Severe Weather Prediction Using Storm Surrogates from an Ensemble Forecasting System , 2016 .

[96]  Robert Sharman,et al.  Nature of Aviation Turbulence , 2016 .

[97]  S. E. Haupt,et al.  A regime-dependent artificial neural network technique for short-range solar irradiance forecasting , 2016 .

[98]  Florian Pappenberger,et al.  Automation and human expertise in operational river forecasting , 2016 .

[99]  John K. Williams,et al.  Adaptable Regression Method for Ensemble Consensus Forecasting , 2016, AAAI.

[100]  Kristin M. Calhoun,et al.  Multi-Radar Multi-Sensor (MRMS) Severe Weather and Aviation Products: Initial Operating Capabilities , 2016 .

[101]  Sue Ellen Haupt,et al.  Variable Generation Power Forecasting as a Big Data Problem , 2017, IEEE Transactions on Sustainable Energy.