HESS Opinions: Deep learning as a promising avenue toward knowledge discovery in water sciences

Abstract. Recently, deep learning (DL) has emerged as a revolutionary and versatile tool transforming industry applications and generating new and improved capabilities for scientific discovery and model building. The adoption of DL in water science has so far been gradual, but the related fields are now ripe for breakthroughs. This paper proposes that DL-based methods can open up a viable, complementary avenue toward knowledge discovery in hydrologic sciences. In the new avenue, machine-learning algorithms present competing hypotheses that are consistent with data for scientists to further evaluate. Interrogative studies are invoked to interpret DL models. In addition, we lay out several opinions shared by authors: (1) deep learning may bring forth transformative progress to the field of hydrology due to its ability to assimilate big data and identify commonalities and differences; (2) The community may benefit greatly from a variety of shared datasets and open competitions; (3) Big hydrologic data can be obtained via various ways including data compilation and working with citizen scientists, which offers the co-benefits of education and stakeholder engagement; (4) Water sciences, and hydrology in particular, offer a unique set of challenges that can, in turn, stimulate advances in machine learning; and (5) An urgent need for research is hydrology-customized methods for interpreting knowledge extracted by deep learning.

[1]  Eric Laloy,et al.  Training‐Image Based Geostatistical Inversion Using a Spatial Generative Adversarial Neural Network , 2017, ArXiv.

[2]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Stefano Ermon,et al.  Monitoring Ethiopian Wheat Fungus with Satellite Imagery and Deep Feature Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[5]  Wenyi Huang,et al.  Detecting Arbitrary Oriented Text in the Wild with a Visual Attention Model , 2016, ACM Multimedia.

[6]  Ali Farhadi,et al.  Deep Classifiers from Image Tags in the Wild , 2015, MMCommons '15.

[7]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  B. Frey,et al.  Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning , 2015, Nature Biotechnology.

[9]  Michael Durand,et al.  Assessing the potential global extent of SWOT river discharge observations , 2014 .

[10]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[11]  A. Ihler,et al.  Precipitation Identification with Bispectral Satellite Information Using Deep Learning Approaches , 2017 .

[12]  Eric Laloy,et al.  Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network , 2017, 1710.09196.

[13]  Volkmar Frinken,et al.  Mode Detection in Online Handwritten Documents Using BLSTM Neural Networks , 2012, 2012 International Conference on Frontiers in Handwriting Recognition.

[14]  S. Sorooshian,et al.  Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks , 1997 .

[15]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  Xiao Yang,et al.  Prolongation of SMAP to Spatiotemporally Seamless Coverage of Continental U.S. Using a Deep Learning Neural Network , 2017, 1707.06611.

[17]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[18]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[19]  Geoffrey Zweig,et al.  Achieving Human Parity in Conversational Speech Recognition , 2016, ArXiv.

[20]  Kuolin Hsu,et al.  A Two-Stage Deep Neural Network Framework for Precipitation Estimation from Bispectral Satellite Information , 2018 .

[21]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[22]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[23]  Peter A. Troch,et al.  Climate-vegetation-soil interactions and long-term hydrologic partitioning: signatures of catchment co-evolution , 2013, Hydrology and Earth System Sciences.

[24]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[25]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Fei-Fei Li,et al.  Visualizing and Understanding Recurrent Networks , 2015, ArXiv.

[27]  Keith Beven,et al.  TOPMODEL : a critique. , 1997 .

[28]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[29]  Kuolin Hsu,et al.  Improved streamflow forecasting using self-organizing radial basis function artificial neural networks , 2004 .

[30]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[31]  Thomas M. Breuel,et al.  High-Performance OCR for Printed English and Fraktur Using LSTM Networks , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[32]  Li-Chiu Chang,et al.  Regional flood inundation nowcast using hybrid SOM and dynamic neural networks , 2014 .

[33]  Martin J. Blunt,et al.  Reconstruction of three-dimensional porous media using generative adversarial neural networks , 2017, Physical review. E.

[34]  Peter A. Troch,et al.  The future of hydrology: An evolving science for a changing world , 2010 .

[35]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[36]  Duo Zhang,et al.  Use Long Short-Term Memory to Enhance Internet of Things for Combined Sewer Overflow Monitoring , 2018 .

[37]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[38]  Prabhat,et al.  Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets , 2016, ArXiv.

[39]  Chen Sun,et al.  Revisiting Unreasonable Effectiveness of Data in Deep Learning Era , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[40]  Stefano Ermon,et al.  Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data , 2017, AAAI.

[41]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[42]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[43]  A. Ihler,et al.  A Deep Neural Network Modeling Framework to Reduce Bias in Satellite Precipitation Products , 2016 .

[44]  Joaquín González-Rodríguez,et al.  Automatic language identification using long short-term memory recurrent neural networks , 2014, INTERSPEECH.

[45]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[46]  Atul K. Jain,et al.  Global patterns of drought recovery , 2015, Nature.

[47]  Sujay V. Kumar,et al.  Benchmarking NLDAS-2 Soil Moisture and Evapotranspiration to Separate Uncertainty Contributions. , 2016, Journal of hydrometeorology.

[48]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[49]  Emanuele Strano,et al.  Modeling Urbanization Patterns with Generative Adversarial Networks , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[50]  Kuolin Hsu,et al.  Artificial Neural Network Modeling of the Rainfall‐Runoff Process , 1995 .

[51]  Maosheng Zhao,et al.  Improvements to a MODIS global terrestrial evapotranspiration algorithm , 2011 .

[52]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[53]  Jiancheng Shi,et al.  The Soil Moisture Active Passive (SMAP) Mission , 2010, Proceedings of the IEEE.

[54]  Ce Zhang,et al.  Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features , 2016, Nature Communications.

[55]  S. Swenson,et al.  Accuracy of GRACE mass estimates , 2006 .