A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists

Deep learning (DL), a new generation of artificial neural network research, has transformed industries, daily lives, and various scientific disciplines in recent years. DL represents significant progress in the ability of neural networks to automatically engineer problem‐relevant features and capture highly complex data distributions. I argue that DL can help address several major new and old challenges facing research in water sciences such as interdisciplinarity, data discoverability, hydrologic scaling, equifinality, and needs for parameter regionalization. This review paper is intended to provide water resources scientists and hydrologists in particular with a simple technical overview, transdisciplinary progress update, and a source of inspiration about the relevance of DL to water. The review reveals that various physical and geoscientific disciplines have utilized DL to address data challenges, improve efficiency, and gain scientific insights. DL is especially suited for information extraction from image‐like data and sequential data. Techniques and experiences presented in other disciplines are of high relevance to water research. Meanwhile, less noticed is that DL may also serve as a scientific exploratory tool. A new area termed AI neuroscience, where scientists interpret the decision process of deep networks and derive insights, has been born. This budding subdiscipline has demonstrated methods including correlation‐based analysis, inversion of network‐extracted features, reduced‐order approximations by interpretable models, and attribution of network decisions to inputs. Moreover, DL can also use data to condition neurons that mimic problem‐specific fundamental organizing units, thus revealing emergent behaviors of these units. Vast opportunities exist for DL to propel advances in water sciences.

[1]  Vaishak Belle,et al.  Proceedings of The Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) , 2017, AAAI 2017.

[2]  Ying Liu,et al.  Geological Disaster Recognition on Optical Remote Sensing Images Using Deep Learning , 2016 .

[3]  Davide Castelvecchi,et al.  Artificial intelligence called in to tackle LHC data deluge , 2015, Nature.

[4]  Murugesu Sivapalan,et al.  Scale issues in hydrological modelling: A review , 1995 .

[5]  Surya Ganguli,et al.  On the Expressive Power of Deep Neural Networks , 2016, ICML.

[6]  Dit-Yan Yeung,et al.  Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model , 2017, NIPS.

[7]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[8]  Eric Laloy,et al.  Training‐Image Based Geostatistical Inversion Using a Spatial Generative Adversarial Neural Network , 2017, ArXiv.

[9]  Shie-Yui Liong,et al.  Rainfall and runoff forecasting with SSA-SVM approach , 2001 .

[10]  Chaopeng Shen,et al.  The introspective may achieve more: Enhancing existing Geoscientific models with native-language emulated structural reflection , 2018, Comput. Geosci..

[11]  Lutz Prechelt,et al.  Early Stopping - But When? , 2012, Neural Networks: Tricks of the Trade.

[12]  Philippe Richaume,et al.  Soil moisture retrieval from SMOS observations using neural networks , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[13]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[14]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[15]  Zhi-Hua Zhou,et al.  NeC4.5: Neural Ensemble Based C4.5 , 2004, IEEE Trans. Knowl. Data Eng..

[16]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[17]  Eric Laloy,et al.  Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network , 2017, 1710.09196.

[18]  N. Chang,et al.  Short-term streamflow forecasting with global climate change implications – A comparative study between genetic programming and neural network models , 2008 .

[19]  Aytac Guven,et al.  Linear genetic programming for time-series modelling of daily flow rate , 2009 .

[20]  Zoubin Ghahramani,et al.  Unsupervised Learning , 2003, Advanced Lectures on Machine Learning.

[21]  Francesco Laio,et al.  Toward the camera rain gauge , 2015 .

[22]  Ernesto Costa,et al.  Dynamic Limits for Bloat Control: Variations on Size and Depth , 2004, GECCO.

[23]  Christian Borgs,et al.  Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes , 2016, Proceedings of the National Academy of Sciences.

[24]  Frank S. Marzano,et al.  A Neural Networks–Based Fusion Technique to Estimate Half-Hourly Rainfall Estimates at 0.1° Resolution from Satellite Passive Microwave and Infrared Data , 2004 .

[25]  Holger R. Maier,et al.  Input determination for neural network models in water resources applications. Part 2. Case study: forecasting salinity in a river , 2005 .

[26]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[27]  Ananthram Swami,et al.  Crafting adversarial input sequences for recurrent neural networks , 2016, MILCOM 2016 - 2016 IEEE Military Communications Conference.

[28]  Karsten Schulz,et al.  Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks , 2018, Hydrology and Earth System Sciences.

[29]  Gunnar Rätsch,et al.  Advanced Lectures on Machine Learning , 2004, Lecture Notes in Computer Science.

[30]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[31]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[32]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[33]  A. Ihler,et al.  A Deep Neural Network Modeling Framework to Reduce Bias in Satellite Precipitation Products , 2016 .

[34]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Abhinav Vishnu,et al.  How Much Chemistry Does a Deep Neural Network Need to Know to Make Accurate Predictions? , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[36]  Chaopeng Shen,et al.  Improving Budyko curve‐based estimates of long‐term water partitioning using hydrologic signatures from GRACE , 2016 .

[37]  Jiancheng Shi,et al.  The Soil Moisture Active Passive (SMAP) Mission , 2010, Proceedings of the IEEE.

[38]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[39]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[40]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[41]  Yoshua Bengio,et al.  Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[43]  Chong-Yu Xu,et al.  Downscaling GCMs using the Smooth Support Vector Machine method to predict daily precipitation in the Hanjiang Basin , 2010 .

[44]  Vijay S. Pande,et al.  Atomic Convolutional Networks for Predicting Protein-Ligand Binding Affinity , 2017, ArXiv.

[45]  Günter Klambauer,et al.  DeepTox: Toxicity Prediction using Deep Learning , 2016, Front. Environ. Sci..

[46]  Patrick T. Komiske,et al.  Deep learning in color: towards automated quark/gluon jet discrimination , 2016, Journal of High Energy Physics.

[47]  Tim R. McVicar,et al.  Global‐scale regionalization of hydrologic model parameters , 2016 .

[48]  Suzanna Becker,et al.  Unsupervised Learning Procedures for Neural Networks , 1991, Int. J. Neural Syst..

[49]  Hongwei Liu,et al.  Convolutional Neural Network With Data Augmentation for SAR Target Recognition , 2016, IEEE Geoscience and Remote Sensing Letters.

[50]  Haytham Assem,et al.  Urban Water Flow and Water Level Prediction Based on Deep Learning , 2017, ECML/PKDD.

[51]  Paresh Chandra Deka,et al.  Support vector machine applications in the field of hydrology: A review , 2014, Appl. Soft Comput..

[52]  Dongryeol Ryu,et al.  Characterization of footprint‐scale surface soil moisture variability using Gaussian and beta distribution functions during the Southern Great Plains 1997 (SGP97) hydrology experiment , 2005 .

[53]  Alfred O. Hero,et al.  Estimating epistemic and aleatory uncertainties during hydrologic modeling: An information theoretic approach , 2013 .

[54]  M. Çimen,et al.  Estimation of daily suspended sediments using support vector machines , 2008 .

[55]  Xi Chen,et al.  Uncertainty analysis of a semi-distributed hydrologic model based on a Gaussian Process emulator , 2018, Environ. Model. Softw..

[56]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[57]  Manolis Kellis,et al.  Deep learning for regulatory genomics , 2015, Nature Biotechnology.

[58]  Richard A. Marcum,et al.  Application of deep convolutional neural networks to automatic feature/object detection in high resolution remote sensing imagery , 2017 .

[59]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[60]  Murugesu Sivapalan,et al.  Socio-hydrologic drivers of the pendulum swing between agricultural development and environmental health: a case study from Murrumbidgee River basin, Australia , 2014 .

[61]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[62]  Li Li,et al.  Bypassing the Kohn-Sham equations with machine learning , 2016, Nature Communications.

[63]  Peter A. Troch,et al.  Catchment coevolution: A useful framework for improving predictions of hydrological change? , 2015 .

[64]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[65]  Pierre Baldi,et al.  The dropout learning algorithm , 2014, Artif. Intell..

[66]  Ole Winther,et al.  Convolutional LSTM Networks for Subcellular Localization of Proteins , 2015, AlCoB.

[67]  Samuel Greengard GPUs reshape computing , 2016, Commun. ACM.

[68]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[69]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[70]  M. Balamurugan,et al.  Genetic programming based monthly groundwater level forecast models with uncertainty quantification , 2016, Modeling Earth Systems and Environment.

[71]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[72]  Samuel Ritter,et al.  Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study , 2017, ICML.

[73]  Filipe Aires,et al.  Soil moisture retrieval from AMSR-E and ASCAT microwave observation synergy. Part 2: Product evaluation , 2017 .

[74]  Paul Voosen,et al.  The AI detectives. , 2017, Science.

[75]  Uwe Stilla,et al.  Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[76]  Hoshin Vijai Gupta,et al.  Systematic Bias in Land Surface Models , 2007 .

[77]  Günter Blöschl,et al.  Socio-hydrology: conceptualising human-flood interactions , 2013 .

[78]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[79]  William B. Claster,et al.  Deep Learning with Convolutional Neural Networks , 2020, Mathematics and Programming for Machine Learning with R.

[80]  Fei-Fei Li,et al.  Visualizing and Understanding Recurrent Networks , 2015, ArXiv.

[81]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[82]  Leslie Pack Kaelbling,et al.  Generalization in Deep Learning , 2017, ArXiv.

[83]  Xiao Xiang Zhu,et al.  Deep learning in remote sensing: a review , 2017, ArXiv.

[84]  James G. Lyons,et al.  Improving prediction of secondary structure, local backbone angles, and solvent accessible surface area of proteins by iterative deep learning , 2015, Scientific Reports.

[85]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[86]  Chuntian Cheng,et al.  Using support vector machines for long-term discharge prediction , 2006 .

[87]  Xiao Yang,et al.  Prolongation of SMAP to Spatiotemporally Seamless Coverage of Continental U.S. Using a Deep Learning Neural Network , 2017, 1707.06611.

[88]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[89]  Chaopeng Shen,et al.  Full‐flow‐regime storage‐streamflow correlation patterns provide insights into hydrologic functioning over the continental US , 2017 .

[90]  Laurence Perreault Levasseur,et al.  Fast automated analysis of strong gravitational lenses with convolutional neural networks , 2017, Nature.

[91]  S. L. Yang,et al.  Downstream sedimentary and geomorphic impacts of the Three Gorges Dam on the Yangtze River , 2014 .

[92]  J. Jawitz,et al.  Doing ecohydrology backward: Inferring wetland flow and hydroperiod from landscape patterns , 2017 .

[93]  Beck Hylke,et al.  Global-scale regionalization of hydrologic model parameters , 2016 .

[94]  T. Schorr The spring of hope. , 1973, The American journal of nursing.

[95]  S. Sorooshian,et al.  Evaluation of PERSIANN system satellite-based estimates of tropical rainfall , 2000 .

[96]  Lior Wolf,et al.  A Dynamic Convolutional Layer for short rangeweather prediction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[97]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[98]  Vijay S. Pande,et al.  Low Data Drug Discovery with One-Shot Learning , 2016, ACS central science.

[99]  Yang Hong,et al.  Estimating a-priori kinematic wave model parameters based on regionalization for flash flood forecasting in the Conterminous United States , 2016 .

[100]  Ivo D. Dinov,et al.  Deep learning for neural networks , 2018 .

[101]  Eric F. Wood,et al.  POLARIS: A 30-meter probabilistic soil series map of the contiguous United States , 2016 .

[102]  Albert J. Valocchi,et al.  Data-driven methods to improve baseflow prediction of a regional groundwater model , 2015, Comput. Geosci..

[103]  C. Sivapragasam,et al.  Genetic programming approach for flood routing in natural channels , 2008 .

[104]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[105]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[106]  H. Akaike A new look at the statistical model identification , 1974 .

[107]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[108]  Luke de Oliveira,et al.  Jet-images — deep learning edition , 2015, Journal of High Energy Physics.

[109]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[110]  Regina Barzilay,et al.  Convolutional Embedding of Attributed Molecular Graphs for Physical Property Prediction , 2017, J. Chem. Inf. Model..

[111]  Vlado Menkovski,et al.  Understanding Anatomy Classification Through Visualization , 2016, ArXiv.

[112]  Ryosuke Shibasaki,et al.  Estimating crop yields with deep learning and remotely sensed data , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[113]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[114]  Olaf Kolditz,et al.  Surface‐subsurface model intercomparison: A first set of benchmark results to diagnose integrated hydrology and feedbacks , 2014 .

[115]  Prabhat,et al.  Revealing Fundamental Physics from the Daya Bay Neutrino Experiment Using Deep Neural Networks , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[116]  Duo Zhang,et al.  Use Long Short-Term Memory to Enhance Internet of Things for Combined Sewer Overflow Monitoring , 2018 .

[117]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[118]  Yoshua Bengio,et al.  A Closer Look at Memorization in Deep Networks , 2017, ICML.

[119]  P. Baldi,et al.  Searching for exotic particles in high-energy physics with deep learning , 2014, Nature Communications.

[120]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[121]  Pierre Gentine,et al.  Could Machine Learning Break the Convection Parameterization Deadlock? , 2018, Geophysical Research Letters.

[122]  Hyun-Woo Lee,et al.  Deep neural networks for wild fire detection with unmanned aerial vehicle , 2017, 2017 IEEE International Conference on Consumer Electronics (ICCE).

[123]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[124]  Eric F. Wood,et al.  Scale Dependence and Scale Invariance in Hydrology: Scale Analyses for Land-Surface Hydrology , 1998 .

[125]  A. Kalra,et al.  Estimating soil moisture using remote sensing data: A machine learning approach , 2010 .

[126]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[127]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[128]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[129]  Tom White,et al.  Generative Adversarial Networks: An Overview , 2017, IEEE Signal Processing Magazine.

[130]  O. Kisi,et al.  SVM, ANFIS, regression and climate based models for reference evapotranspiration modeling using limited climatic data in a semi-arid highland environment , 2012 .

[131]  Sangram Ganguly,et al.  DeepSD: Generating High Resolution Climate Change Projections through Single Image Super-Resolution , 2017, KDD.

[132]  Vijay S. Pande,et al.  Massively Multitask Networks for Drug Discovery , 2015, ArXiv.

[133]  Deep Transfer Learning: A new deep learning glitch classification method for advanced LIGO , 2017, ArXiv.

[134]  J. McCallum,et al.  Nonparametric estimation of groundwater residence time distributions: What can environmental tracer data tell us about groundwater residence time? , 2014 .

[135]  Geoffrey E. Hinton,et al.  Learning representations by back-propagation errors, nature , 1986 .

[136]  Y. Hong,et al.  Precipitation Estimation from Remotely Sensed Imagery Using an Artificial Neural Network Cloud Classification System , 2004 .

[137]  Dragan Savic,et al.  A Genetic Programming Approach to Rainfall-Runoff Modelling , 1999 .

[138]  Madan K. Jha,et al.  Efficacy of neural network and genetic algorithm techniques in simulating spatio‐temporal fluctuations of groundwater , 2015 .

[139]  Mac McKee,et al.  Multi-time scale stream flow predictions: The support vector machines approach , 2006 .

[140]  Ying Fan,et al.  Hydrologic regulation of plant rooting depth , 2017, Proceedings of the National Academy of Sciences.

[141]  Ole-Christoffer Granmo,et al.  Deep Convolutional Neural Networks for Fire Detection in Images , 2017, EANN.

[142]  Jiancheng Shi,et al.  The Future of Earth Observation in Hydrology. , 2017, Hydrology and earth system sciences.

[143]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[144]  J. T. Childers,et al.  A Particle Consistent with the Higgs Boson Observed with the ATLAS Detector at the Large Hadron Collider , 2012, Science.

[145]  Kuldip K. Paliwal,et al.  Predicting backbone Cα angles and dihedrals from protein sequences by stacked sparse auto‐encoder deep neural network , 2014, J. Comput. Chem..

[146]  Michael C. Mozer,et al.  A Focused Backpropagation Algorithm for Temporal Pattern Recognition , 1989, Complex Syst..

[147]  Nikolaos Doulamis,et al.  Deep supervised learning for hyperspectral data classification through convolutional neural networks , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[148]  P Baldi,et al.  Enhanced Higgs boson to τ(+)τ(-) search with deep learning. , 2014, Physical review letters.

[149]  Hayaru Shouno,et al.  Analysis of Dropout Learning Regarded as Ensemble Learning , 2016, ICANN.

[150]  Shiming Xiang,et al.  Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks , 2014, IEEE Geoscience and Remote Sensing Letters.

[151]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[152]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[153]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[154]  Feng Liu,et al.  Modeling spatio-temporal distribution of soil moisture by deep learning-based cellular automata model , 2016, Journal of Arid Land.

[155]  Stefano Ermon,et al.  Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data , 2017, AAAI.

[156]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[157]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[158]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[159]  Prabhat,et al.  Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets , 2016, ArXiv.

[160]  Amin Elshorbagy,et al.  Prediction of hourly actual evapotranspiration using neural networks, genetic programming, and statistical models , 2010 .

[161]  Vicente Julián,et al.  Rainfall Prediction: A Deep Learning Approach , 2016, HAIS.

[162]  Kuolin Hsu,et al.  Effective and Efficient Modeling for Streamflow Forecasting , 2000 .

[163]  Chen Sun,et al.  Revisiting Unreasonable Effectiveness of Data in Deep Learning Era , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[164]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[165]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[166]  Saurabh Suradhaniwar,et al.  Gaussian process based spatial modeling of soil moisture for dense soil moisture sensing network , 2017, 2017 6th International Conference on Agro-Geoinformatics.

[167]  M. Sivapalan,et al.  Characterizing hydrologic change through catchment classification , 2014 .

[168]  H. B. Barlow,et al.  Unsupervised Learning , 1989, Neural Computation.

[169]  Subhransu Maji,et al.  Automatic Image Annotation using Deep Learning Representations , 2015, ICMR.

[170]  T. Lømo Nobel Prize in Physiology or Medicine 2014. , 2014, Tidsskrift for Den Norske Laegeforening.

[171]  R. S. Govindaraju,et al.  Artificial Neural Networks in Hydrology , 2010 .

[172]  J. Hartigan Direct Clustering of a Data Matrix , 1972 .

[173]  Chuntian Cheng,et al.  A comparison of performance of several artificial intelligence methods for forecasting monthly discharge time series , 2009 .

[174]  Kuolin Hsu,et al.  Self‐organizing linear output map (SOLO): An artificial neural network suitable for hydrologic modeling and analysis , 2002 .

[175]  Zaher Mundher Yaseen,et al.  Artificial intelligence based models for stream-flow forecasting: 2000-2015 , 2015 .

[176]  Jingjing Xie,et al.  Daily reservoir inflow forecasting using multiscale deep feature learning with hybrid models , 2016 .

[177]  Kuolin Hsu,et al.  Neural Error Regression Diagnosis (NERD): A Tool for Model Bias Identification and Prognostic Data Assimilation , 2006 .

[178]  V. Natarajan,et al.  [The Nobel Prize in physiology or medicine]. , 1998, Tidsskrift for den Norske laegeforening : tidsskrift for praktisk medicin, ny raekke.

[179]  Holger R. Maier,et al.  The effect of internal parameters and geometry on the performance of back-propagation neural networks: an empirical study , 1998 .

[180]  Holger R. Maier,et al.  Input determination for neural network models in water resources applications. Part 1—background and methodology , 2005 .

[181]  Emanuele Strano,et al.  Modeling Urbanization Patterns with Generative Adversarial Networks , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[182]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[183]  M. Schaap,et al.  ROSETTA: a computer program for estimating soil hydraulic parameters with hierarchical pedotransfer functions , 2001 .

[184]  Yann LeCun,et al.  Pedestrian Detection with Unsupervised Multi-stage Feature Learning , 2012, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[185]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[186]  A. Scolobig,et al.  Insights from socio-hydrology modelling on dealing with flood risk - Roles of collective memory, risk-taking attitude and trust , 2013 .

[187]  Pierre Baldi,et al.  Efficient antihydrogen detection in antimatter physics by deep learning , 2017, ArXiv.

[188]  Zhen Li,et al.  Understanding Hidden Memories of Recurrent Neural Networks , 2017, 2017 IEEE Conference on Visual Analytics Science and Technology (VAST).

[189]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[190]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[191]  David G. Tarboton,et al.  An overview of current applications, challenges, and future trends in distributed process-based models in hydrology , 2016 .

[192]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[193]  Jiaolong Xu,et al.  Deep Convolutional Neural Networks for Forest Fire Detection , 2016 .

[194]  Serge Andrianov,et al.  Comparison of Regularization Methods for ImageNet Classification with Deep Convolutional Neural Networks , 2014 .

[195]  David J. Schwab,et al.  An exact mapping between the Variational Renormalization Group and Deep Learning , 2014, ArXiv.

[196]  Ohad Shamir,et al.  The Power of Depth for Feedforward Neural Networks , 2015, COLT.

[197]  Dana H. Ballard,et al.  Modular Learning in Neural Networks , 1987, AAAI.

[198]  Joseph Gomes,et al.  MoleculeNet: a benchmark for molecular machine learning† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c7sc02664a , 2017, Chemical science.

[199]  Amin Elshorbagy,et al.  Estimating Saturated Hydraulic Conductivity Using Genetic Programming , 2007 .

[200]  Ronald,et al.  Learning representations by backpropagating errors , 2004 .

[201]  Razvan Pascanu,et al.  Vector-based navigation using grid-like representations in artificial agents , 2018, Nature.

[202]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[203]  G. Kasieczka,et al.  Deep-learned Top Tagging using Lorentz Invariance and Nothing Else , 2017 .

[204]  M. Scheffler,et al.  The face of crystals: insightful classification using deep learning , 2017 .

[205]  MohammadSajjad Khan,et al.  Application of Support Vector Machine in Lake Water Level Prediction , 2006 .

[206]  K. P. Sudheer,et al.  Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions , 2010, Environ. Model. Softw..

[207]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[208]  Hareton K. N. Leung,et al.  A Deep-Learning Based Precipitation Forecasting Approach Using Multiple Environmental Factors , 2017, 2017 IEEE International Congress on Big Data (BigData Congress).

[209]  Kuolin Hsu,et al.  Bias Adjustment of Satellite Precipitation Estimation Using Ground-Based Measurement: A Case Study Evaluation over the Southwestern United States , 2009 .

[210]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[211]  George M. Church,et al.  Biclustering of Expression Data , 2000, ISMB.

[212]  Jia Liu,et al.  Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images , 2016 .

[213]  Pierre Baldi,et al.  Deep Architectures and Deep Learning in Chemoinformatics: The Prediction of Aqueous Solubility for Drug-Like Molecules , 2013, J. Chem. Inf. Model..

[214]  Praveen Kumar,et al.  A data mining approach for understanding topographic control on climate-induced inter-annual vegetation variability over the United States , 2005 .

[215]  Ajmal Mian,et al.  Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey , 2018, IEEE Access.

[216]  M. Ek,et al.  Hyperresolution global land surface modeling: Meeting a grand challenge for monitoring Earth's terrestrial water , 2011 .

[217]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.

[218]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[219]  Geoffrey E. Hinton,et al.  Distilling a Neural Network Into a Soft Decision Tree , 2017, CEx@AI*IA.

[220]  Jean-Luc Dugelay,et al.  Face aging with conditional generative adversarial networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[221]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[222]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[223]  Daniel Kifer,et al.  Unifying Adversarial Training Algorithms with Flexible Deep Data Gradient Regularization , 2016, ArXiv.

[224]  Tonio Ball,et al.  Deep learning with convolutional neural networks for decoding and visualization of EEG pathology , 2017, 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[225]  A. Ihler,et al.  Precipitation Identification with Bispectral Satellite Information Using Deep Learning Approaches , 2017 .

[226]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[227]  V. V. Srinivas,et al.  Downscaling of precipitation for climate change scenarios: A support vector machine approach , 2006 .

[228]  Jianlin Cheng,et al.  A Deep Learning Network Approach to ab initio Protein Secondary Structure Prediction , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[229]  Gregor Kasieczka,et al.  Deep-learned Top Tagging with a Lorentz Layer , 2017, SciPost Physics.

[230]  K. Lee,et al.  A comparative study of artificial neural networks and support vector machines for predicting groundwater levels in a coastal aquifer , 2011 .

[231]  Murugesu Sivapalan,et al.  A prototype framework for models of socio-hydrology: identification of key feedback loops and parameterisation approach , 2014 .

[232]  Dmitri Kavetski,et al.  A unified approach for process‐based hydrologic modeling: 2. Model implementation and case studies , 2015 .

[233]  Vijay S. Pande,et al.  Computational Modeling of β-Secretase 1 (BACE-1) Inhibitors Using Ligand Based Approaches , 2016, J. Chem. Inf. Model..

[234]  Qing Liu,et al.  Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[235]  Dejing Dou,et al.  A deep learning approach for human behavior prediction with explanations in health social networks: social restricted Boltzmann machine (SRBM+) , 2016, Social Network Analysis and Mining.

[236]  Byunghan Lee,et al.  Deep learning in bioinformatics , 2016, Briefings Bioinform..

[237]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[238]  A. Norman Redlich,et al.  Redundancy Reduction as a Strategy for Unsupervised Learning , 1993, Neural Computation.

[239]  K. Tai,et al.  A Computational Intelligence-Based Genetic Programming Approach for the Simulation of Soil Water Retention Curves , 2014, Transport in Porous Media.

[240]  Luke de Oliveira,et al.  Image Processing, Computer Vision, and Deep Learning: new approaches to the analysis and physics interpretation of LHC events , 2016, Journal of Physics: Conference Series.

[241]  Erhardt Barth,et al.  Recurrent Dropout without Memory Loss , 2016, COLING.

[242]  Santiago Velasco-Forero,et al.  Deep learning for studies of galaxy morphology , 2016, Astroinformatics.

[243]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[244]  Vijay S. Pande,et al.  MoleculeNet: a benchmark for molecular machine learning , 2017, Chemical science.

[245]  Ingmar Kanitscheider,et al.  Training recurrent networks to generate hypotheses about how the brain solves hard navigation problems , 2016, NIPS.

[246]  Simon A. Wagner,et al.  SAR ATR by a combination of convolutional neural network and support vector machines , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[247]  Abhinav Vishnu,et al.  Deep learning for computational chemistry , 2017, J. Comput. Chem..

[248]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[249]  Graham W. Taylor,et al.  Adaptive deconvolutional networks for mid and high level feature learning , 2011, 2011 International Conference on Computer Vision.

[250]  Jesús S. Aguilar-Ruiz,et al.  Biclustering on expression data: A review , 2015, J. Biomed. Informatics.

[251]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .

[252]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[253]  Jean-Raynald de Dreuzy,et al.  Prospective Interest of Deep Learning for Hydrological Inference , 2017, Ground water.

[254]  S. Pacala,et al.  A METHOD FOR SCALING VEGETATION DYNAMICS: THE ECOSYSTEM DEMOGRAPHY MODEL (ED) , 2001 .

[255]  Upmanu Lall,et al.  Depletion and response of deep groundwater to climate-induced pumping variability , 2017 .

[256]  J. Melack,et al.  The fan of influence of streams and channel feedbacks to simulated land surface water and carbon dynamics , 2016 .

[257]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[258]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[259]  Elahe Fallah-Mehdipour,et al.  Prediction and simulation of monthly groundwater levels by genetic programming , 2013 .

[260]  Peter A. Troch,et al.  The future of hydrology: An evolving science for a changing world , 2010 .

[261]  H. Maier,et al.  The Use of Artificial Neural Networks for the Prediction of Water Quality Parameters , 1996 .

[262]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[263]  Stefano Ermon,et al.  Monitoring Ethiopian Wheat Fungus with Satellite Imagery and Deep Feature Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[264]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[265]  Sujay V. Kumar,et al.  Benchmarking NLDAS-2 Soil Moisture and Evapotranspiration to Separate Uncertainty Contributions. , 2016, Journal of hydrometeorology.

[266]  Jie Geng,et al.  High-Resolution SAR Image Classification via Deep Convolutional Autoencoders , 2015, IEEE Geoscience and Remote Sensing Letters.

[267]  Nagiza F. Samatova,et al.  Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[268]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[269]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition , 2012 .

[270]  Alexander M. Rush,et al.  Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks , 2016, ArXiv.

[271]  Wei-keng Liao,et al.  Toward enhanced understanding and projections of climate extremes using physics-guided data mining techniques , 2014 .

[272]  G. Blöschl,et al.  The Growth of Hydrological Understanding: Technologies, Ideas, and Societal Needs Shape the Field , 2017 .

[273]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[274]  Eric F. Wood,et al.  An efficient calibration method for continental‐scale land surface modeling , 2008 .

[275]  Alexander Y. Sun,et al.  Monthly streamflow forecasting using Gaussian Process Regression , 2014 .

[276]  R. Katz,et al.  Daily spatiotemporal precipitation simulation using latent and transformed Gaussian processes , 2012 .

[277]  Y. Yao,et al.  On Early Stopping in Gradient Descent Learning , 2007 .

[278]  K-R Müller,et al.  SchNet - A deep learning architecture for molecules and materials. , 2017, The Journal of chemical physics.

[279]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[280]  Yashesh Gaur,et al.  Robust Speech Recognition Using Generative Adversarial Networks , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[281]  Makarand Deo,et al.  Real time wave forecasting using neural networks , 1998 .

[282]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[283]  D. Lawrence,et al.  Improving the representation of hydrologic processes in Earth System Models , 2015 .

[284]  Olof Mogren,et al.  C-RNN-GAN: Continuous recurrent neural networks with adversarial training , 2016, ArXiv.

[285]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[286]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[287]  F. Aires,et al.  A new neural network approach including first guess for retrieval of atmospheric water vapor, cloud liquid water path, surface temperature, and emissivities over land from satellite microwave observations , 2001 .

[288]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[289]  Kwang-Tsao Shao,et al.  A data-mining framework for exploring the multi-relation between fish species and water quality through self-organizing map. , 2017, The Science of the total environment.

[290]  Ximing Cai,et al.  Understanding and managing the food-energy-water nexus – opportunities for water resources research , 2018 .

[291]  B. Frey,et al.  Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning , 2015, Nature Biotechnology.

[292]  David Morgan,et al.  Deep convolutional neural networks for ATR from SAR imagery , 2015, Defense + Security Symposium.

[293]  I-Fan Chang,et al.  Support vector regression for real-time flood stage forecasting , 2006 .

[294]  G. Rätsch A Brief Introduction into Machine Learning , 2004 .

[295]  Christian W. Dawson,et al.  Hydrological modelling using artificial neural networks , 2001 .

[296]  M. Scheffler,et al.  Insightful classification of crystal structures using deep learning , 2017, Nature Communications.

[297]  M. P.R.,et al.  A METHOD FOR SCALING VEGETATION DYNAMICS: THE ECOSYSTEM DEMOGRAPHY MODEL (ED) , 2022 .

[298]  J. Kirchner Catchments as simple dynamical systems: Catchment characterization, rainfall‐runoff modeling, and doing hydrology backward , 2009 .

[299]  Prabhat,et al.  ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events , 2016, NIPS.

[300]  P. Vahle,et al.  A convolutional neural network neutrino event classifier , 2016, ArXiv.

[301]  David A. Clausi,et al.  Sea Ice Concentration Estimation During Melt From Dual-Pol SAR Scenes Using Deep Convolutional Neural Networks: A Case Study , 2016, IEEE Transactions on Geoscience and Remote Sensing.