The importance of short lag-time in the runoff forecasting model based on long short-term memory

Abstract It is still very challenging to enhance the accuracy and stability of daily runoff forecasts, especially several days ahead, owing to the non-linearity of the forecasted processes. Here, we hypothesize that short lag-time has a significant impact on forecasting results. Thus, we incorporate short previous time steps into long short-term memory (LSTM) and develop the Self-Attentive Long Short-Term Memory (SA-LSTM). In SA-LSTM, the self-attention mechanism is used to model interdependencies within short previous time steps. SA-LSTM is evaluated at eight runoff datasets. The experimental results demonstrate that, compared with state-of-art benchmark models, SA-LSTM achieves the best performance. The RMSEs of SA-LSTM are at least 2.3% smaller than that of the second best model at the seventh day. The NSEs and NSE_In of SA-LSTM are at least 4.6% and 6.4% higher than those of the second best model at the seventh day. Furthermore, SA-LSTM also surpasses the baseline methods for base, mean and peak flows. The superiority of SA-LSTM can be attributed to its exploitation of information in short lag-time.

[1]  Roberto Revelli,et al.  A comparison of nonlinear flood forecasting methods , 2003 .

[2]  Rajib Maity,et al.  Potential of support vector regression for prediction of monthly streamflow using endogenous property , 2010 .

[3]  Emmanouil N. Anagnostou,et al.  Machine Learning–Based Blending of Satellite and Reanalysis Precipitation Datasets: A Multiregional Tropical Complex Terrain Evaluation , 2019, Journal of Hydrometeorology.

[4]  Jianfeng Wu,et al.  Streamflow and rainfall forecasting by two long short-term memory-based models , 2020 .

[5]  J. Sabo,et al.  Landscape heterogeneity and hydrological processes: a review of landscape-based hydrological models , 2018, Landscape Ecology.

[6]  John Quilty,et al.  Addressing the incorrect usage of wavelet-based hydrological and water resources forecasting models for real-world applications with best practices and a new forecasting framework , 2018, Journal of Hydrology.

[7]  Aleksei Romanenko,et al.  Hyper-parameter Optimization of Multi-attention Recurrent Neural Network for Battery State-of-Charge Forecasting , 2019, EPIA.

[8]  Ying Zhao,et al.  The incorrect usage of singular spectral analysis and discrete wavelet transform in hybrid models to predict hydrological time series , 2017 .

[9]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[10]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[11]  K. Beven.  Rainfall-Runoff Modelling: The Primer , 2012 .

[12]  M. Palmer,et al.  Linkages between flow regime, biota, and ecosystem processes: Implications for river restoration , 2019, Science.

[13]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[14]  Mac McKee,et al.  BASIN SCALE WATER MANAGEMENT AND FORECASTING USING ARTIFICIAL NEURAL NETWORKS 1 , 2005 .

[15]  Milan Cisty,et al.  Flow Prediction Versus Flow Simulation Using Machine Learning Algorithms , 2018, MLDM.

[16]  P. Claps,et al.  Changing climate both increases and decreases European river floods , 2019, Nature.

[17]  Günter Blöschl,et al.  Flood fatalities in Africa: From diagnosis to mitigation , 2010 .

[18]  Shie-Yui Liong,et al.  FLOOD STAGE FORECASTING WITH SUPPORT VECTOR MACHINES 1 , 2002 .

[19]  Emmanouil N. Anagnostou,et al.  A nonparametric statistical technique for combining global precipitation datasets: development and hydrological evaluation over the Iberian Peninsula , 2017 .

[20]  Andreas Spanias,et al.  Attend and Diagnose: Clinical Time Series Analysis using Attention Models , 2017, AAAI.

[21]  Keith W. Hipel,et al.  Forecasting monthly riverflow time series , 1985 .

[22]  Daniela M. Witten,et al.  An Introduction to Statistical Learning: with Applications in R , 2013 .

[23]  D. G. Watts,et al.  Application of Linear Random Models to Four Annual Streamflow Series , 1970 .

[24]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[25]  H. Savenije,et al.  The importance of aspect for modelling the hydrological response in a glacier catchment in Central Asia , 2017 .

[26]  A. Langousis,et al.  A Brief Review of Random Forests for Water Scientists and Practitioners and Their Recent History in Water Resources , 2019, Water.

[27]  Fu Chao Liu,et al.  Hybrid forecasting model for non-stationary daily runoff series: A case study in the Han River Basin, China , 2019, Journal of Hydrology.

[28]  Hubert H. G. Savenije,et al.  Testing the realism of a topography driven model (FLEX-Topo) in the nested catchments of the Upper Heihe, China , 2013 .

[29]  Zhanya Xu,et al.  A hybrid support vector regression framework for streamflow forecast , 2019, Journal of Hydrology.

[30]  Demetris Koutsoyiannis,et al.  Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes , 2019, Stochastic Environmental Research and Risk Assessment.

[31]  Dawen Yang,et al.  A continental scale hydrological model using the distributed approach and its application to Asia , 2003 .

[32]  Marie-Amélie Boucher,et al.  A Stochastic Data‐Driven Ensemble Forecasting Framework for Water Resources: A Case Study Using Ensemble Members Derived From a Database of Deterministic Wavelet‐Based Models , 2019, Water Resources Research.

[33]  J. Sheffield,et al.  Anthropogenic shift towards higher risk of flash drought over China , 2019, Nature Communications.

[34]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[35]  Mac McKee,et al.  Multi-time scale stream flow predictions: The support vector machines approach , 2006 .

[36]  J. Ball,et al.  Rainfall models for catchment simulation , 2005 .

[37]  K. Stahl,et al.  Technical note: Representing glacier geometry changes in a semi-distributed hydrological model , 2018 .

[38]  S. Attinger,et al.  Multiscale parameter regionalization of a grid‐based hydrologic model at the mesoscale , 2010 .

[39]  Y. Najjar,et al.  Predicting catchment flow in a semi‐arid region via an artificial neural network technique , 2004 .

[40]  X. Wen,et al.  A comparative study of artificial neural network, adaptive neuro fuzzy inference system and support vector machine for forecasting river flow in the semiarid mountain region , 2014 .

[41]  Ozgur Kisi,et al.  Streamflow Forecasting Using Different Artificial Neural Network Algorithms , 2007 .

[42]  Zaher Mundher Yaseen,et al.  RBFNN versus FFNN for daily river flow forecasting at Johor River, Malaysia , 2015, Neural Computing and Applications.

[43]  Tommy S. W. Wong,et al.  Runoff forecasting for an asphalt plane by Artificial Neural Networks and comparisons with kinematic wave and autoregressive moving average models , 2011 .

[44]  A. Shamseldin.  Application of a neural network technique to rainfall-runoff modelling , 1997 .

[45]  Masud Rana,et al.  Effects of Activation Functions and Optimizers on Stock Price Prediction using LSTM Recurrent Networks , 2019, CSAI.

[46]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[47]  Zhongmin Liang,et al.  Combination of Multiple Data-Driven Models for Long-Term Monthly Runoff Predictions Based on Bayesian Model Averaging , 2019, Water Resources Management.

[48]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[49]  Murugesu Sivapalan,et al.  The Growth of Hydrological Understanding: Technologies, Ideas, and Societal Needs Shape the Field , 2017 .

[50]  Soroosh Sorooshian,et al.  Hydrological modelling and the water cycle : coupling the atmospheric and hydrological models , 2008 .

[51]  Skipper Seabold,et al.  Statsmodels: Econometric and Statistical Modeling with Python , 2010, SciPy.

[52]  M. Taqqu,et al.  Fractionally differenced ARIMA models applied to hydrologic time series: Identification, estimation, and simulation , 1997 .

[53]  V. Singh,et al.  Streamflow forecast uncertainty evolution and its effect on real-time reservoir operation , 2016 .

[54]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[55]  Xing-wei Chen,et al.  Impacts of reservoir operations on multi-scale correlations between hydrological drought and meteorological drought , 2018, Journal of Hydrology.

[56]  J. McDonnell,et al.  A decade of Predictions in Ungauged Basins (PUB)—a review , 2013 .

[57]  Conceptual Hydrological Models , 2017 .

[58]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[59]  Heng Zhang,et al.  Dynamic Streamflow Simulation via Online Gradient-Boosted Regression Tree , 2019, Journal of Hydrologic Engineering.

[60]  Vijay P. Singh,et al.  Regionalization and spatial changing properties of droughts across the Pearl River basin, China , 2012 .

[61]  Bruno CaprileIRST.  MODEL CALIBRATION , 1997 .

[62]  H. Savenije,et al.  A simple topography-driven and calibration-free runoff generation module , 2019, Hydrology and Earth System Sciences.

[63]  Qian Zhu,et al.  Integration of a Parsimonious Hydrological Model with Recurrent Neural Networks for Improved Streamflow Forecasting , 2018, Water.

[64]  Maurice Schmeits,et al.  Comparing Area Probability Forecasts of (Extreme) Local Precipitation Using Parametric and Machine Learning Statistical Postprocessing Methods , 2018, Monthly Weather Review.

[65]  Zhiyong Liu,et al.  A Framework for Exploring Joint Effects of Conditional Factors on Compound Floods , 2018 .

[66]  C. Birkel,et al.  Advancing tracer‐aided rainfall–runoff modelling: a review of progress, problems and unrealised potential , 2015 .

[67]  Haytham Assem,et al.  Urban Water Flow and Water Level Prediction Based on Deep Learning , 2017, ECML/PKDD.

[68]  Chaopeng Shen,et al.  A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists , 2017, Water Resources Research.

[69]  C. Birkel,et al.  Modelling storage‐driven connectivity between landscapes and riverscapes: towards a simple framework for long‐term ecohydrological assessment , 2016 .

[70]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[71]  András Bárdossy,et al.  Modeling of the effect of land use changes on the runoff generation of a river basin through parameter regionalization of a watershed model , 2004 .

[72]  A. Danandeh Mehr,et al.  Successive-station monthly streamflow prediction using different artificial neural network algorithms , 2015, International Journal of Environmental Science and Technology.

[73]  Zhongbo Yu,et al.  On continental-scale hydrologic simulations with a coupled hydrologic model , 2006 .

[74]  Jose D. Salas,et al.  APPROACHES TO MULTIVARIATE MODELING OF WATER RESOURCES TIME SERIES1 , 1985 .