Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks

Rainfall-runoff modelling is one of the key challenges in the field of hydrology. Various approaches exist, ranging from physically based over conceptual to fully data driven models. In this paper, we propose a novel data driven approach, using the Long-Short-Term-Memory (LSTM) network, a special type of recurrent neural networks. The advantage of the LSTM is its ability to learn long-term dependencies between the provided input and output of the network, which are essential for modelling storage effects in e.g. catchments with snow influence. We use 241 catchments of the freely available CAMELS data set to 5 test our approach and also compare the results to the well-known Sacramento Soil Moisture Accounting Model (SAC-SMA) coupled with the Snow-17 snow routine. We also show the potential of the LSTM as a regional hydrological model, in which one model predicts the discharge for a variety of catchments. In our last experiment, we show the possibility to transfer process understanding, learned at regional scale, to individual catchments and thereby increasing model performance when compared to a LSTM trained only on the data of single catchments. Using this approach, we were able to achieve better model performance 10 as the SAC-SMA + Snow-17, which underlines the potential of the LSTM for hydrological modelling applications.

[1]  Michael P. Rogers.  Python Tutorial , 2009 .

[2]  Eric A. Anderson,et al.  National Weather Service river forecast system: snow accumulation and ablation model , 1973 .

[3]  J. Thielen,et al.  The European Flood Alert System – Part 1: Concept and development , 2008 .

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  A. Ihler,et al.  A Deep Neural Network Modeling Framework to Reduce Bias in Satellite Precipitation Products , 2016 .

[6]  Eric A. Rosenberg,et al.  A Long-Term Hydrologically Based Dataset of Land Surface Fluxes and States for the Conterminous United States: Update and Extensions* , 2002 .

[7]  J. V. Soares,et al.  HAND, a new terrain descriptor using SRTM-DEM: Mapping terra-firme rainforest environments in Amazonia , 2008 .

[8]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[9]  L. Hay,et al.  Hydrometeorological dataset for the contiguous USA , 2014 .

[10]  Kuolin Hsu,et al.  HESS Opinions: Incubating deep-learning-powered hydrologic science advances as a community , 2018, Hydrology and Earth System Sciences.

[11]  K. Beven.  Rainfall-Runoff Modelling: The Primer , 2012 .

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Peter C. Young,et al.  Data-based mechanistic modelling and the rainfall-flow non-linearity. , 1994 .

[14]  Luis Samaniego,et al.  The evolution of process-based hydrologic models: Historical challenges and the collective quest for physical realism. , 2017, Hydrology and earth system sciences.

[15]  F. P. Kapinos,et al.  Hydrologic unit maps , 1987 .

[16]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[17]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[18]  Martyn P. Clark,et al.  Development of a large-sample watershed-scale hydrometeorological data set for the contiguous USA: data set characteristics and assessment of regional variability in hydrologic model performance , 2014 .

[19]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[20]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Dimitri P. Solomatine,et al.  Data-Driven Modelling: Concepts, Approaches and Experiences , 2009 .

[22]  Kuolin Hsu,et al.  Application of a recurrent neural network to rainfall-runoff modeling , 1997 .

[23]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[24]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[25]  Maosheng Zhao,et al.  Improvements to a MODIS global terrestrial evapotranspiration algorithm , 2011 .

[26]  S. Sorooshian,et al.  Shuffled complex evolution approach for effective and efficient global minimization , 1993 .

[27]  H. Halff,et al.  Predicting Runoff from Rainfall Using Neural Networks , 1993 .

[28]  Luiz Paulo Lopes Fávero,et al.  Estimation , 2019, Data Science for Business and Decision Making.

[29]  M. Ek,et al.  Hyperresolution global land surface modeling: Meeting a grand challenge for monitoring Earth's terrestrial water , 2011 .

[30]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[31]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  M. Sivapalan.  Prediction in ungauged basins: a grand challenge for theoretical hydrology , 2003 .

[34]  Marvin N. Wright,et al.  SoilGrids250m: Global gridded soil information based on machine learning , 2017, PloS one.

[35]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[36]  S. Running,et al.  Global products of vegetation leaf area and fraction absorbed PAR from year one of MODIS data , 2002 .

[37]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[38]  Peter Norvig,et al.  The Unreasonable Effectiveness of Data , 2009, IEEE Intelligent Systems.

[39]  Hoshin Vijai Gupta,et al.  Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling , 2009 .

[40]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[41]  J. Nash,et al.  River flow forecasting through conceptual models part I — A discussion of principles☆ , 1970 .

[42]  Martyn P. Clark,et al.  The CAMELS data set: catchment attributes and meteorology for large-sample studies , 2017 .

[43]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[44]  Jan Vanderborght,et al.  Proof of concept of regional scale hydrologic simulations at hydrologic resolution utilizing massively parallel computer resources , 2010 .

[45]  Jean-Raynald de Dreuzy,et al.  Prospective Interest of Deep Learning for Hydrological Inference , 2017, Ground water.

[46]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[47]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[48]  Yang Yang,et al.  Deep Learning Scaling is Predictable, Empirically , 2017, ArXiv.

[49]  Jonathan Tompson,et al.  Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation , 2014, NIPS.

[50]  M. Ye,et al.  Developing a Long Short-Term Memory (LSTM) based model for predicting water table depth in agricultural areas , 2018, Journal of Hydrology.

[51]  Jimson Mathew,et al.  Hydrological Data Driven Modelling: A Case Study Approach , 2015 .

[52]  P. Coulibaly,et al.  Two decades of anarchy? Emerging themes and outstanding challenges for neural network river forecasting , 2012 .

[53]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[54]  G. Blöschl,et al.  Regionalization methods in rainfall-runoff modelling using large catchment samples. , 2006 .

[55]  S. Attinger,et al.  Multiscale parameter regionalization of a grid‐based hydrologic model at the mesoscale , 2010 .

[56]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[57]  Erwin Zehe,et al.  A review of regionalisation for continuous streamflow simulation , 2011 .

[58]  R. Freeze,et al.  Blueprint for a physically-based, digitally-simulated hydrologic response model , 1969 .

[59]  Keith Beven,et al.  Regionalization as a learning process , 2009 .

[60]  Wes McKinney,et al.  Data Structures for Statistical Computing in Python , 2010, SciPy.

[61]  J. Kirchner.  Getting the right answers for the right reasons: Linking measurements, analyses, and models to advance the science of hydrology , 2006 .

[62]  T. Sathish,et al.  River Flow Forecasting using Recurrent Neural Networks , 2004 .

[63]  Duo Zhang,et al.  Use Long Short-Term Memory to Enhance Internet of Things for Combined Sewer Overflow Monitoring , 2018 .

[64]  Keith Beven,et al.  Linking parameters across scales: Subgrid parameterizations and scale dependent hydrological models. , 1995 .

[65]  Mathew Herrnegger,et al.  Continuous hydrological modelling in the context of real time flood forecasting in alpine Danube tributary catchments , 2008 .

[66]  Haytham Assem,et al.  Urban Water Flow and Water Level Prediction Based on Deep Learning , 2017, ECML/PKDD.

[67]  C. Luce.  Runoff Prediction in Ungauged Basins: Synthesis Across Processes, Places and Scales , 2014 .

[68]  Hoshin Vijai Gupta,et al.  A process‐based diagnostic approach to model evaluation: Application to the NWS distributed hydrologic model , 2008 .

[69]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[70]  B. Arheimer,et al.  Development and testing of the HYPE (Hydrological Predictions for the Environment) water quality model for different spatial scales , 2010 .

[71]  K. Mo,et al.  Continental-scale water and energy flux analysis and validation for the North American Land Data Assimilation System project phase 2 (NLDAS-2): 1. Intercomparison and application of model products , 2012 .

[72]  Mathew Herrnegger,et al.  Adjustment of spatio-temporal precipitation patterns in a high Alpine environment , 2018 .

[73]  D. Lettenmaier,et al.  A Long-Term Hydrologically Based Dataset of Land Surface Fluxes and States for the Conterminous United States* , 2002 .

[74]  K. Schulz,et al.  Hydrological modelling in the anthroposphere: predicting local runoff in a heavily modified high-alpine catchment , 2018, Journal of Mountain Science.

[75]  Kuolin Hsu,et al.  HESS Opinions: Deep learning as a promising avenue toward knowledge discovery in water sciences , 2018 .

[76]  Xiao Yang,et al.  Prolongation of SMAP to Spatiotemporally Seamless Coverage of Continental U.S. Using a Deep Learning Neural Network , 2017, 1707.06611.