HESS Opinions: Incubating deep-learning-powered hydrologic science advances as a community

Abstract. Recently, deep learning (DL) has emerged as a revolutionary and versatile tool transforming industry applications and generating new and improved capabilities for scientific discovery and model building. The adoption of DL in hydrology has so far been gradual, but the field is now ripe for breakthroughs. This paper suggests that DL-based methods can open up a complementary avenue toward knowledge discovery in hydrologic sciences. In the new avenue, machine-learning algorithms present competing hypotheses that are consistent with data. Interrogative methods are then invoked to interpret DL models for scientists to further evaluate. However, hydrology presents many challenges for DL methods, such as data limitations, heterogeneity and co-evolution, and the general inexperience of the hydrologic field with DL. The roadmap toward DL-powered scientific advances will require the coordinated effort from a large community involving scientists and citizens. Integrating process-based models with DL models will help alleviate data limitations. The sharing of data and baseline models will improve the efficiency of the community as a whole. Open competitions could serve as the organizing events to greatly propel growth and nurture data science education in hydrology, which demands a grassroots collaboration. The area of hydrologic DL presents numerous research opportunities that could, in turn, stimulate advances in machine learning as well.

[1]  S LewMichael,et al.  Deep learning for visual understanding , 2016 .

[2]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Stefano Ermon,et al.  Monitoring Ethiopian Wheat Fungus with Satellite Imagery and Deep Feature Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Wenyi Huang,et al.  Detecting Arbitrary Oriented Text in the Wild with a Visual Attention Model , 2016, ACM Multimedia.

[5]  Ashish Sharma,et al.  Development of a computationally efficient semi-distributed hydrologic modeling application for soil moisture, lateral flow and runoff simulation , 2016, Environ. Model. Softw..

[6]  T. Wagener,et al.  It takes a community to raise a hydrologist: the Modular Curriculum for Hydrologic Advancement (MOCHA) , 2012 .

[7]  R. S. Govindaraju,et al.  Artificial Neural Networks in Hydrology , 2010 .

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[10]  Geoffrey Zweig,et al.  Achieving Human Parity in Conversational Speech Recognition , 2016, ArXiv.

[11]  B. Frey,et al.  Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning , 2015, Nature Biotechnology.

[12]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  A. Ihler,et al.  A Deep Neural Network Modeling Framework to Reduce Bias in Satellite Precipitation Products , 2016 .

[14]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[15]  Joaquín González-Rodríguez,et al.  Automatic language identification using long short-term memory recurrent neural networks , 2014, INTERSPEECH.

[16]  Emanuele Strano,et al.  Modeling Urbanization Patterns with Generative Adversarial Networks , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[17]  Sabine Attinger,et al.  Accelerating advances in continental domain hydrologic modeling , 2015 .

[18]  M. Hipsey,et al.  “Panta Rhei—Everything Flows”: Change in hydrology and society—The IAHS Scientific Decade 2013–2022 , 2013 .

[19]  R Govindaraju,et al.  ARTIFICIAL NEURAL NETWORKS IN HYDROLOGY: II, HYDROLOGIC APPLICATIONS , 2000 .

[20]  Christopher D. Manning,et al.  Advances in natural language processing , 2015, Science.

[21]  John R. Josephson,et al.  Abductive inference : computation, philosophy, technology , 1994 .

[22]  K. P. Sudheer,et al.  Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions , 2010, Environ. Model. Softw..

[23]  J. McDonnell,et al.  A decade of Predictions in Ungauged Basins (PUB)—a review , 2013 .

[24]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[25]  Xiao Yang,et al.  Prolongation of SMAP to Spatiotemporally Seamless Coverage of Continental U.S. Using a Deep Learning Neural Network , 2017, 1707.06611.

[26]  Luca Delle Monache,et al.  Probabilistic Weather Prediction with an Analog Ensemble , 2013 .

[27]  Kuolin Hsu,et al.  A Two-Stage Deep Neural Network Framework for Precipitation Estimation from Bispectral Satellite Information , 2018 .

[28]  Kuolin Hsu,et al.  Artificial Neural Network Modeling of the Rainfall‐Runoff Process , 1995 .

[29]  Mohamed M. Morsy,et al.  Modeling urban coastal flood severity from crowd-sourced flood reports using Poisson regression and Random Forest , 2018 .

[30]  Maosheng Zhao,et al.  Improvements to a MODIS global terrestrial evapotranspiration algorithm , 2011 .

[31]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[32]  Constantin Junk,et al.  Predictor-weighting strategies for probabilistic wind power forecasting with an analog ensemble , 2015 .

[33]  D. Entekhabi,et al.  Soil Moisture Active Passive (SMAP) Project Calibration and Validation for the L2/3_SM_P , 2015 .

[34]  Vlado Menkovski,et al.  Understanding Anatomy Classification Through Visualization , 2016, ArXiv.

[35]  Pierre Gentine,et al.  Could Machine Learning Break the Convection Parameterization Deadlock? , 2018, Geophysical Research Letters.

[36]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Chaopeng Shen,et al.  Full‐flow‐regime storage‐streamflow correlation patterns provide insights into hydrologic functioning over the continental US , 2017 .

[38]  Kathleen J. Lehman,et al.  Anatomy of an Enduring Gender Gap: The Evolution of Women’s Participation in Computer Science , 2017 .

[39]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[40]  Antonio Torralba,et al.  Anticipating Visual Representations from Unlabeled Video , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[42]  Eric Laloy,et al.  Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network , 2017, 1710.09196.

[43]  Martha C. Anderson,et al.  Precipitation Estimation from Remotely Sensed Information Using Articial Neural Networks , 2012 .

[44]  Volkmar Frinken,et al.  Mode Detection in Online Handwritten Documents Using BLSTM Neural Networks , 2012, 2012 International Conference on Frontiers in Handwriting Recognition.

[45]  Andreas Dengel,et al.  EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[46]  M. Schaap,et al.  ROSETTA: a computer program for estimating soil hydraulic parameters with hierarchical pedotransfer functions , 2001 .

[47]  Tegan Maharaj,et al.  Deep Nets Don't Learn via Memorization , 2017, ICLR.

[48]  Atul K. Jain,et al.  Global patterns of drought recovery , 2015, Nature.

[49]  Erik Skau,et al.  Fusing Heterogeneous Data: A Case for Remote Sensing and Social Media , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Thomas M. Breuel,et al.  High-Performance OCR for Printed English and Fraktur Using LSTM Networks , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[51]  Li-Chiu Chang,et al.  Regional flood inundation nowcast using hybrid SOM and dynamic neural networks , 2014 .

[52]  Karsten Schulz,et al.  Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks , 2018, Hydrology and Earth System Sciences.

[53]  Martin J. Blunt,et al.  Reconstruction of three-dimensional porous media using generative adversarial neural networks , 2017, Physical review. E.

[54]  Karsten Schulz,et al.  Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks , 2018, Hydrology and Earth System Sciences.

[55]  James H. Faghmous,et al.  A Big Data Guide to Understanding Climate Change: The Case for Theory-Guided Data Science , 2014, Big Data.

[56]  Paul Voosen,et al.  The AI detectives. , 2017, Science.

[57]  Holger R. Maier,et al.  Data-driven modelling approaches for socio-hydrology: opportunities and challenges within the Panta Rhei Science Plan , 2016 .

[58]  Jeffery S. Horsburgh,et al.  HydroShare: Sharing Diverse Environmental Data Types and Models as Social Objects with Application to the Hydrology Domain , 2016 .

[59]  Soroosh Sorooshian,et al.  Classification and regression tree (CART) analysis for indicator bacterial concentration prediction for a Californian coastal area. , 2010, Water science and technology : a journal of the International Association on Water Pollution Research.

[60]  Jiancheng Shi,et al.  The Future of Earth Observation in Hydrology. , 2017, Hydrology and earth system sciences.

[61]  Keith Beven,et al.  TOPMODEL : a critique. , 1997 .

[62]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[63]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[64]  Justin Basilico,et al.  DeepQA Jeopardy! Gamification: A Machine-Learning Perspective , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[65]  Heiga Zen,et al.  Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[66]  Victor Zlotnicki,et al.  Time‐variable gravity from GRACE: First results , 2004 .

[67]  William Aspray,et al.  Women and Underrepresented Minorities in Computing: A Historical and Social Study , 2016 .

[68]  Eric Laloy,et al.  Training‐Image Based Geostatistical Inversion Using a Spatial Generative Adversarial Neural Network , 2017, ArXiv.

[69]  Yann Kerr,et al.  The Soil Moisture and Ocean Salinity Mission - An Overview , 2008, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium.

[70]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[71]  Jens Hartmann,et al.  A glimpse beneath earth's surface: GLobal HYdrogeology MaPS (GLHYMPS) of permeability and porosity , 2014 .

[72]  R. Stull,et al.  Kalman Filter and Analog Schemes to Postprocess Numerical Weather Predictions , 2011 .

[73]  Ali Farhadi,et al.  Deep Classifiers from Image Tags in the Wild , 2015, MMCommons '15.

[74]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[75]  Michael Durand,et al.  Assessing the potential global extent of SWOT river discharge observations , 2014 .

[76]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[77]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[78]  Ian Goodfellow,et al.  Generative adversarial networks , 2020, Commun. ACM.

[79]  Kuolin Hsu,et al.  Self‐organizing linear output map (SOLO): An artificial neural network suitable for hydrologic modeling and analysis , 2002 .

[80]  Sangram Ganguly,et al.  DeepSD: Generating High Resolution Climate Change Projections through Single Image Super-Resolution , 2017, KDD.

[81]  Kathleen Joelle Lehman Courting the Uncommitted: A Mixed-Methods Study of Undecided Students in Introductory Computer Science Courses , 2017 .

[82]  Haytham Assem,et al.  Urban Water Flow and Water Level Prediction Based on Deep Learning , 2017, ECML/PKDD.

[83]  S. Swenson,et al.  Accuracy of GRACE mass estimates , 2006 .

[84]  Dmitri Kavetski,et al.  A unified approach for process‐based hydrologic modeling: 1. Modeling concept , 2015 .

[85]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[86]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[87]  Lucy Marshall,et al.  Creativity, Uncertainty, and Automated Model Building , 2017, Ground water.

[88]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[89]  Li-Chiu Chang,et al.  Exploring the spatio-temporal interrelation between groundwater and surface water by using the self-organizing maps , 2018 .

[90]  P Baldi,et al.  Enhanced Higgs boson to τ(+)τ(-) search with deep learning. , 2014, Physical review letters.

[91]  Chaopeng Shen,et al.  A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists , 2017, Water Resources Research.

[92]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[93]  Akira Ishii,et al.  Methods of the Water-Energy-Food Nexus , 2015 .

[94]  Duo Zhang,et al.  Use Long Short-Term Memory to Enhance Internet of Things for Combined Sewer Overflow Monitoring , 2018 .

[95]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[96]  Keith Beven,et al.  Changing ideas in hydrology — The case of physically-based models , 1989 .

[97]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[98]  Chen Sun,et al.  Revisiting Unreasonable Effectiveness of Data in Deep Learning Era , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[99]  Stefano Ermon,et al.  Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data , 2017, AAAI.

[100]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[101]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[102]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[103]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[104]  Kuolin Hsu,et al.  Improved streamflow forecasting using self-organizing radial basis function artificial neural networks , 2004 .

[105]  Kuolin Hsu,et al.  Neural Error Regression Diagnosis (NERD): A Tool for Model Bias Identification and Prognostic Data Assimilation , 2006 .

[106]  Prabhat,et al.  Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets , 2016, ArXiv.

[107]  Razvan Pascanu,et al.  Vector-based navigation using grid-like representations in artificial agents , 2018, Nature.

[108]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[109]  Trevor Darrell,et al.  Generating Visual Explanations , 2016, ECCV.

[110]  Kei Hiroi,et al.  FloodEye: Real-time flash flood prediction system for urban complex water flow , 2016, 2016 IEEE SENSORS.

[111]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[112]  Leslie Pack Kaelbling,et al.  Generalization in Deep Learning , 2017, ArXiv.

[113]  Venkatesh Merwade,et al.  Moving university hydrology education forward with community-based geoinformatics, data and modeling resources , 2012 .

[114]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[115]  Nagiza F. Samatova,et al.  Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[116]  Peter A. Troch,et al.  The future of hydrology: An evolving science for a changing world , 2010 .

[117]  Yoshua Bengio,et al.  A Closer Look at Memorization in Deep Networks , 2017, ICML.

[118]  A. Ihler,et al.  Precipitation Identification with Bispectral Satellite Information Using Deep Learning Approaches , 2017 .

[119]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[120]  Francesco Laio,et al.  Toward the camera rain gauge , 2015 .

[121]  Sujay V. Kumar,et al.  Benchmarking NLDAS-2 Soil Moisture and Evapotranspiration to Separate Uncertainty Contributions. , 2016, Journal of hydrometeorology.

[122]  Xiao Xiang Zhu,et al.  Deep learning in remote sensing: a review , 2017, ArXiv.

[123]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[124]  Günter Blöschl,et al.  Hydrologic synthesis: Across processes, places, and scales , 2006 .

[125]  Fei-Fei Li,et al.  Visualizing and Understanding Recurrent Networks , 2015, ArXiv.

[126]  William Aspray,et al.  Women and Underrepresented Minorities in Computing , 2016, History of Computing.

[127]  John H. Grove,et al.  Soil-landscape modeling across a physiographic region : Topographic patterns and model transportability , 2006 .

[128]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[129]  null null,et al.  Artificial Neural Networks in Hydrology. II: Hydrologic Applications , 2000 .

[130]  Daniel Kifer,et al.  Unifying Adversarial Training Algorithms with Flexible Deep Data Gradient Regularization , 2016, ArXiv.

[131]  Peter A. Troch,et al.  Climate-vegetation-soil interactions and long-term hydrologic partitioning: signatures of catchment co-evolution , 2013, Hydrology and Earth System Sciences.

[132]  Hoshin Vijai Gupta,et al.  Systematic Bias in Land Surface Models , 2007 .

[133]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[134]  S. Running,et al.  MODIS Leaf Area Index (LAI) And Fraction Of Photosynthetically Active Radiation Absorbed By Vegetation (FPAR) Product , 1999 .

[135]  Jiancheng Shi,et al.  The Soil Moisture Active Passive (SMAP) Mission , 2010, Proceedings of the IEEE.

[136]  Eric F. Wood,et al.  Deriving global parameter estimates for the Noah land surface model using FLUXNET and machine learning , 2016 .

[137]  Abhinav Vishnu,et al.  Deep learning for computational chemistry , 2017, J. Comput. Chem..

[138]  Ce Zhang,et al.  Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features , 2016, Nature Communications.