TRU-NET: A Deep Learning Approach to High Resolution Prediction of Rainfall

Climate models (CM) are used to evaluate the impact of climate change on the risk of floods and strong precipitation events. However, these numerical simulators have difficulties representing precipitation events accurately, mainly due to limited spatial resolution when simulating multi-scale dynamics in the atmosphere. To improve the prediction of high resolution precipitation we apply a Deep Learning (DL) approach using an input of CM simulations of the model fields (weather variables) that are more predictable than local precipitation. To this end, we present TRU-NET (Temporal Recurrent U-Net), an encoder-decoder model featuring a novel 2D cross attention mechanism between contiguous convolutional-recurrent layers to effectively model multi-scale spatio-temporal weather processes. We use a conditional-continuous loss function to capture the zero-skewed %extreme event patterns of rainfall. Experiments show that our model consistently attains lower RMSE and MAE scores than a DL model prevalent in short term precipitation prediction and improves upon the rainfall predictions of a state-of-the-art dynamical weather model. Moreover, by evaluating the performance of our model under various, training and testing, data formulation strategies, we show that there is enough data for our deep learning approach to output robust, high-quality results across seasons and varying regions.

[1]  V. Masson‐Delmotte,et al.  Climate Change and Land: an IPCC special report on climate change, desertification, land degradation, sustainable land management, food security, and greenhouse gas fluxes in terrestrial ecosystems , 2019 .

[2]  Dit-Yan Yeung,et al.  Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model , 2017, NIPS.

[3]  Junjie Yao,et al.  Recurrent Neural Network for Text Classification with Hierarchical Multiscale Dense Connections , 2019, IJCAI.

[4]  Fu-En Yang,et al.  Learning Hierarchical Self-Attention for Video Summarization , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[5]  Liyuan Liu,et al.  On the Variance of the Adaptive Learning Rate and Beyond , 2019, ICLR.

[6]  Pierre Gentine,et al.  Deep learning to represent subgrid processes in climate models , 2018, Proceedings of the National Academy of Sciences.

[7]  Michael Gamon,et al.  Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer , 2018, ArXiv.

[8]  Finale Doshi-Velez,et al.  Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors , 2018, ICML.

[9]  Wang-chun Woo,et al.  Operational Application of Optical Flow Techniques to Radar-Based Rainfall Nowcasting , 2017 .

[10]  Sangram Ganguly,et al.  DeepSD: Generating High Resolution Climate Change Projections through Single Image Super-Resolution , 2017, KDD.

[11]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[12]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[13]  J. Michaelsen,et al.  Use of the gamma distribution to represent monthly rainfall in Africa for drought monitoring applications , 2007 .

[14]  Richard Coe,et al.  A Model Fitting Analysis of Daily Rainfall Data , 1984 .

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Jennifer G. Dy,et al.  Quantifying Uncertainty in Discrete-Continuous and Skewed Data with Bayesian Deep Learning , 2018, KDD.

[17]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[18]  Xiangnan He,et al.  Modeling Extreme Events in Time Series Prediction , 2019, KDD.

[19]  Soukayna Mouatadid,et al.  WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting , 2020, Journal of Advances in Modeling Earth Systems.

[20]  J. Thepaut,et al.  The ERA5 global reanalysis , 2020, Quarterly Journal of the Royal Meteorological Society.

[21]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[22]  W. May Simulation of the variability and extremes of daily rainfall during the Indian summer monsoon for present and future times in a global time-slice experiment , 2004 .

[23]  Ashish Vaswani,et al.  Self-Attention with Relative Position Representations , 2018, NAACL.

[24]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.