Stochastic simulation on reproducing long-term memory of hydroclimatological variables using deep learning model

Abstract Stochastic simulation has been employed for producing long-term records and assessing the impact of climate change on hydrological and climatological variables in the future. However, traditional stochastic simulation of hydroclimatological variables often underestimates the variability and correlation structure of larger timescale due to the preservation of long-term memory. However, the Long Short-Term Memory (LSTM) model, one type of recurrent neural network (RNN), employed in different fields, exhibits a remarkable long-term memory characteristic owing to the recursive hidden and cell states. The current study, therefore, applied the LSTM model to the stochastic simulation of hydroclimatological variables to examine how good the LSTM model can preserve the long-term memory and overcome the drawbacks of conventional time series models. The simulation involved a trigonometric function and the Rossler system as well as real case studies for hydrological and climatological variables. Results showed that the LSTM model reproduced the variability and correlation structure of the larger timescale as well as the key statistics of the original time domain better than the traditional models. The hidden and cell states of the LSTM containing the long-memory and oscillation structure following the observations allows better performance compared to the other tested conventional models. This better representation of the long-term variability can be critical in water manager since future water resources planning and management is highly related with this long-term variability. Thus, it is concluded that the LSTM model can be a potential alternative for the stochastic simulation of hydroclimatological variables. Also, note that another long-term memory model such as Gated Recurrent Unit can be also applicable.

[1]  Upmanu Lall,et al.  A Nearest Neighbor Bootstrap For Resampling Hydrologic Time Series , 1996 .

[2]  R. Deo,et al.  Stream-flow forecasting using extreme learning machines: a case study in a semi-arid region in Iraq , 2016 .

[3]  Demetris Koutsoyiannis,et al.  Univariate Time Series Forecasting of Temperature and Precipitation with a Focus on Machine Learning Algorithms: a Multiple-Case Study from Greece , 2018, Water Resources Management.

[4]  Jose D. Salas,et al.  Stochastic Streamflow Simulation Using SAMS-2003 , 2006 .

[5]  Balaji Rajagopalan,et al.  Statistical Nonparametric Model for Natural Salt Estimation , 2005 .

[6]  Chaopeng Shen,et al.  A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists , 2017, Water Resources Research.

[7]  Ahsan Kareem,et al.  Nonlinear Signal Analysis: Time-Frequency Perspectives , 2007 .

[8]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[9]  Donna M. Rizzo,et al.  Advances in ungauged streamflow prediction using artificial neural networks , 2010 .

[10]  Yuhong Yang,et al.  Cross-validation for selecting a model selection procedure , 2015 .

[11]  Shengzhi Huang,et al.  Monthly streamflow prediction using modified EMD-based support vector machine , 2014 .

[12]  Gilberto Fisch,et al.  The Long-Range Memory and the Fractal Dimension: a Case Study for Alcântara , 2017 .

[13]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[14]  Leonard A. Smith,et al.  Uncertainty dynamics and predictability in chaotic systems , 2007 .

[15]  Robert Leconte,et al.  A daily stochastic weather generator for preserving low-frequency of climate variability , 2010 .

[16]  Jian Zhou,et al.  Water quality prediction method based on LSTM neural network , 2017, 2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE).

[17]  Demetris Koutsoyiannis,et al.  Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes , 2019, Stochastic Environmental Research and Risk Assessment.

[18]  Taha B. M. J. Ouarda,et al.  Stochastic simulation of nonstationary oscillation hydroclimatic processes using empirical mode decomposition , 2012 .

[19]  Taesam Lee,et al.  Nonparametric Simulation of Single-Site Seasonal Streamflows , 2010 .

[20]  Wei Wang,et al.  Dependency-based long short term memory network for drug-drug interaction extraction , 2017, BMC Bioinformatics.

[21]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[22]  J. Wallace,et al.  A Pacific Interdecadal Climate Oscillation with Impacts on Salmon Production , 1997 .

[23]  Yundi Jiang,et al.  An improved method for nonlinear parameter estimation: a case study of the Rössler model , 2016, Theoretical and Applied Climatology.

[24]  O. Rössler An equation for continuous chaos , 1976 .

[25]  Yanbin Yuan,et al.  Monthly runoff forecasting based on LSTM–ALO model , 2018, Stochastic Environmental Research and Risk Assessment.

[26]  J. ...,et al.  Applied modeling of hydrologic time series , 1980 .

[27]  E. Berbery,et al.  Analysis Links Pacific Decadal Variability to Drought and Streamflow in United States , 1999 .

[28]  Aini Hussain,et al.  Erratum to: Daily Forecasting of Dam Water Levels: Comparing a Support Vector Machine (SVM) Model With Adaptive Neuro Fuzzy Inference System (ANFIS) , 2013, Water Resources Management.

[29]  Richard A. Davis,et al.  Simple consistent estimation of the coefficients of a linear filter , 1988 .

[30]  J. Adamowski,et al.  Multi-step streamflow forecasting using data-driven non-linear methods in contrasting climate regimes , 2014 .

[31]  Mohamed M. Morsy,et al.  Forecasting Groundwater Table in a Flood Prone Coastal City with Long Short-term Memory and Recurrent Neural Networks , 2019, Water.

[32]  Taesam Lee Stochastic simulation of precipitation data for preserving key statistics in their original domain and application to climate change analysis , 2016, Theoretical and Applied Climatology.

[33]  Kuolin Hsu,et al.  HESS Opinions: Incubating deep-learning-powered hydrologic science advances as a community , 2018, Hydrology and Earth System Sciences.

[34]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[35]  Martijn J. Booij,et al.  Simulation and forecasting of streamflows using machine learning models coupled with base flow separation , 2018, Journal of Hydrology.

[36]  Bing Li,et al.  Comparison of random forests and other statistical methods for the prediction of lake water level: a case study of the Poyang Lake in China , 2016 .

[37]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[38]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[39]  Taesam Lee,et al.  An enhanced nonparametric streamflow disaggregation model with genetic algorithm , 2010 .

[40]  Jose D. Salas,et al.  Prediction of Extreme Events in Hydrologic Processes that Exhibit Abrupt Shifting Patterns , 2005 .

[41]  J. Salas,et al.  Modeling the Dynamics of Long-Term Variability of Hydroclimatic Processes , 2003 .

[42]  Taesam Lee,et al.  Copula-based stochastic simulation of hydrological data applied to Nile River flows , 2011 .

[43]  H. E. Hurst,et al.  Long-Term Storage Capacity of Reservoirs , 1951 .