A review of radar-based nowcasting of precipitation and applicable machine learning techniques

A 'nowcast' is a type of weather forecast which makes predictions in the very short term, typically less than two hours - a period in which traditional numerical weather prediction can be limited. This type of weather prediction has important applications for commercial aviation; public and outdoor events; and the construction industry, power utilities, and ground transportation services that conduct much of their work outdoors. Importantly, one of the key needs for nowcasting systems is in the provision of accurate warnings of adverse weather events, such as heavy rain and flooding, for the protection of life and property in such situations. Typical nowcasting approaches are based on simple extrapolation models applied to observations, primarily rainfall radar. In this paper we review existing techniques to radar-based nowcasting from environmental sciences, as well as the statistical approaches that are applicable from the field of machine learning. Nowcasting continues to be an important component of operational systems and we believe new advances are possible with new partnerships between the environmental science and machine learning communities.

[1]  Amy McGovern,et al.  Making the Black Box More Transparent: Understanding the Physical Implications of Machine Learning , 2019, Bulletin of the American Meteorological Society.

[2]  Patrick Gallinari,et al.  Deep learning for physical processes: incorporating prior scientific knowledge , 2017, ICLR.

[3]  L. Li,et al.  Nowcasting of Motion and Growth of Precipitation with Radar over a Complex Orography , 1995 .

[4]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[5]  Neill E. Bowler,et al.  Development of a precipitation nowcasting algorithm based upon optical flow techniques , 2004 .

[6]  I. Zawadzki,et al.  Predictability of Precipitation from Continental Radar Images. Part III: Operational Nowcasting Implementation (MAPLE) , 2004 .

[7]  Alexander Kann STATEMENT OF GUIDANCE FOR NOWCASTING AND VERY SHORT RANGE FORECASTING (VSRF) , 2018 .

[8]  Yoshua Bengio,et al.  NICE: Non-linear Independent Components Estimation , 2014, ICLR.

[9]  Lior Wolf,et al.  A Dynamic Convolutional Layer for short rangeweather prediction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Rob J. Hyndman,et al.  Boosting multi-step autoregressive forecasts , 2014, ICML.

[11]  Sergey Levine,et al.  Unsupervised Learning for Physical Interaction through Video Prediction , 2016, NIPS.

[12]  Stéphane Laroche,et al.  A Variational Analysis Method for Retrieval of Three-Dimensional Wind Field from Single-Doppler Radar Data , 1994 .

[13]  Alex Graves,et al.  Conditional Image Generation with PixelCNN Decoders , 2016, NIPS.

[14]  W H Hand,et al.  An object‐oriented technique for nowcasting heavy showers and thunderstorms , 2007 .

[15]  Ke Xu,et al.  A Kernel-Based Spatio-Temporal Dynamical Model for Nowcasting Weather Radar Reflectivities , 2005 .

[16]  P. Lang Cell Tracking and Warning Indicators derived from Operational Radar Products , 2001 .

[17]  Yale Song,et al.  Video Prediction with Appearance and Motion Conditions , 2018, ICML.

[18]  D. Schertzer,et al.  Physical modeling and analysis of rain and clouds by anisotropic scaling multiplicative processes , 1987 .

[19]  I. Zawadzki,et al.  Scale-Dependence of the Predictability of Precipitation from Continental Radar Images. Part I: Description of the Methodology , 2002 .

[20]  Jason Hickey,et al.  Machine Learning for Precipitation Nowcasting from Radar Images , 2019, ArXiv.

[21]  Stéphane Laroche,et al.  Retrievals of Horizontal Winds from Single-Doppler Clear-Air Data by Methods of Cross Correlation and Variational Analysis , 1995 .

[22]  Matthias Steiner,et al.  Probabilistic forecasts of mesoscale convective system initiation using the random forest data mining technique , 2016 .

[23]  A. Seed,et al.  STEPS: A probabilistic precipitation forecasting scheme which merges an extrapolation nowcast with downscaled NWP , 2006 .

[24]  A. Bellon,et al.  The Evaluation of Two Years of Real-Time Operation of a Short-Term Precipitation Forecasting Procedure (SHARP). , 1978 .

[25]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[26]  Samy Bengio,et al.  Density estimation using Real NVP , 2016, ICLR.

[27]  Keith A. Browning Review Lecture: Local weather forecasting , 1980, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[28]  Jan Handwerker,et al.  Cell tracking with TRACE3D—a new algorithm , 2002 .

[29]  Antonio Torralba,et al.  Generating Videos with Scene Dynamics , 2016, NIPS.

[30]  Alex Graves,et al.  Video Pixel Networks , 2016, ICML.

[31]  Tilmann Gneiting,et al.  Editorial: Probabilistic forecasting , 2008 .

[32]  Aleksander Madry,et al.  A Classification-Based Study of Covariate Shift in GAN Distributions , 2017, ICML.

[33]  Lindsay J. Bennett,et al.  A review of the initiation of precipitating convection in the United Kingdom , 2006 .

[34]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[35]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[36]  Jianmin Wang,et al.  PredCNN: Predictive Learning with Cascade Convolutions , 2018, IJCAI.

[37]  U. Germann,et al.  A non-stationary stochastic ensemble generator for radar rainfall fields based on the Short-Space Fourier Transform , 2017 .

[38]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[39]  Yingyu Liang,et al.  Generalization and Equilibrium in Generative Adversarial Nets (GANs) , 2017, ICML.

[40]  Steven A. Rutledge,et al.  Nowcasting Storm Initiation and Growth Using GOES-8 and WSR-88D Data , 2003 .

[41]  Sergey Levine,et al.  Stochastic Variational Video Prediction , 2017, ICLR.

[42]  Lei Han,et al.  A Machine Learning Nowcasting Method based on Real-time Reanalysis Data , 2016, ArXiv.

[43]  M. Dixon,et al.  TITAN: Thunderstorm Identification, Tracking, Analysis, and Nowcasting—A Radar-based Methodology , 1993 .

[44]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[45]  Dit-Yan Yeung,et al.  Machine Learning for Spatiotemporal Sequence Forecasting: A Survey , 2018, ArXiv.

[46]  W Schmid,et al.  Short-term risk forecasts of heavy rainfall. , 2002, Water science and technology : a journal of the International Association on Water Pollution Research.

[47]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[48]  Jason Hickey,et al.  MetNet: A Neural Weather Model for Precipitation Forecasting , 2020, ArXiv.

[49]  J. Bech Doppler Radar Observations - Weather Radar, Wind Profiler, Ionospheric Radar, and Other Advanced Applications , 2014 .

[50]  Michael Steinheimer,et al.  Advances in Geosciences Improved nowcasting of precipitation based on convective analysis fields , 2008 .

[51]  A. Bellon,et al.  The use of digital weather radar records for short‐term precipitation forecasting , 1974 .

[52]  A. Seed A Dynamic and Spatial Scaling Approach to Advection Forecasting , 2001 .

[53]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[54]  A. Wiin-Nielsen,et al.  On the Application of Trajectory Methods in Numerical Forecasting , 1959 .

[55]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[56]  Contents , 2012, International Law Reports.

[57]  Honglak Lee,et al.  Action-Conditional Video Prediction using Deep Networks in Atari Games , 2015, NIPS.

[58]  A. Deluca,et al.  Scientific challenges of convective-scale numerical weather prediction , 2017 .

[59]  Clive Pierce,et al.  Formulation and evaluation of a scale decomposition‐based stochastic precipitation nowcast scheme , 2013 .

[60]  Luc Van Gool,et al.  Dynamic Filter Networks , 2016, NIPS.

[61]  Juanzhen Sun,et al.  Use of NWP for Nowcasting Convective Precipitation: Recent Progress and Challenges , 2014 .

[62]  Dara Entekhabi,et al.  Error Propagation of Radar Rainfall Nowcasting Fields through a Fully Distributed Flood Forecasting Model , 2007 .

[63]  Jiajun Wu,et al.  Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks , 2016, NIPS.

[64]  Patrick Willems,et al.  Enhanced object-based tracking algorithm for convective rain storms and cells , 2018 .

[65]  Petr Novák,et al.  CELLTRACK ― Convective cell tracking algorithm and its use for deriving life cycle characteristics , 2009 .

[66]  Colin Raffel,et al.  Towards GAN Benchmarks Which Require Generalization , 2020, ICLR.

[67]  Koray Kavukcuoglu,et al.  Pixel Recurrent Neural Networks , 2016, ICML.

[68]  John K. Williams,et al.  Probabilistic 0–1-h Convective Initiation Nowcasts that Combine Geostationary Satellite Observations and Numerical Weather Prediction Model Data , 2015 .

[69]  Klaus H. Maier-Hein,et al.  A Probabilistic U-Net for Segmentation of Ambiguous Images , 2018, NeurIPS.

[70]  Tage Andersson,et al.  A Model for Probability Nowcasts of Accumulated Precipitation Using Radar , 1991 .

[71]  Neil I. Fox,et al.  A Bayesian Quantitative Precipitation Nowcast Scheme , 2005 .

[72]  I. Zawadzki,et al.  Scale Dependence of the Predictability of Precipitation from Continental Radar Images. Part II: Probability Forecasts , 2004 .

[73]  Rob Fergus,et al.  Stochastic Video Generation with a Learned Prior , 2018, ICML.

[74]  Elizabeth E. Ebert,et al.  Sydney 2000 Forecast Demonstration Project: Convective Storm Nowcasting , 2004 .

[75]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[76]  Petr Novák,et al.  The Czech Hydrometeorological Institute's severe storm nowcasting system , 2007 .

[77]  R. Rinehart,et al.  Three-dimensional storm motion detection by conventional weather radar , 1978, Nature.

[78]  M. D'Apuzzo,et al.  A composite-power-law noise generator , 2008, 2008 IEEE Instrumentation and Measurement Technology Conference.

[79]  Dit-Yan Yeung,et al.  Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model , 2017, NIPS.

[80]  Younghae Do,et al.  Improved rainfall nowcasting using Burgers’ equation , 2020 .

[81]  Daniele Nerini,et al.  Using a 10-Year Radar Archive for Nowcasting Precipitation Growth and Decay: A Probabilistic Machine Learning Approach , 2019, Weather and Forecasting.

[82]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[83]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[84]  Patrick Gallinari,et al.  Learning Dynamical Systems from Partial Observations , 2019, ArXiv.

[85]  J. S. Sawyer A semi-Lagrangian method of solving the vorticity advection equation , 1963 .

[86]  Arthur Witt,et al.  The Storm Cell Identification and Tracking Algorithm: An Enhanced WSR-88D Algorithm , 1998 .

[87]  Cesare Furlanello,et al.  Precipitation Nowcasting with Orographic Enhanced Stacked Generalization: Improving Deep Learning Predictions on Extreme Events , 2020, Atmosphere.

[88]  Juanzhen Sun,et al.  Dynamical and Microphysical Retrieval from Doppler Radar Observations Using a Cloud Model and Its Adjoint. Part I: Model Development and Simulated Data Experiments. , 1997 .

[89]  G. Foote,et al.  Determination of the Boundary Layer Airflow from a Single Doppler Radar , 1990 .