Realistic River Image Synthesis Using Deep Generative Adversarial Networks

In this paper, we demonstrated a practical application of realistic river image generation using deep learning. Specifically, we explored a generative adversarial network (GAN) model capable of generating high-resolution and realistic river images that can be used to support modeling and analysis in surface water estimation, river meandering, wetland loss, and other hydrological research studies. First, we have created an extensive repository of overhead river images to be used in training. Second, we incorporated the Progressive Growing GAN (PGGAN), a network architecture that iteratively trains smaller-resolution GANs to gradually build up to a very high resolution to generate high quality (i.e., 1,024 × 1,024) synthetic river imagery. With simpler GAN architectures, difficulties arose in terms of exponential increase of training time and vanishing/exploding gradient issues, which the PGGAN implementation seemed to significantly reduce. The results presented in this study show great promise in generating high-quality images and capturing the details of river structure and flow to support hydrological modeling and research.

[1]  Muhammed Sit,et al.  Short-term Hourly Streamflow Prediction with Graph Convolutional GRU Networks , 2021, ArXiv.

[2]  Muhammed Sit,et al.  A Comprehensive Review of Deep Learning Applications in Hydrology and Water Resources , 2020, Water science and technology : a journal of the International Association on Water Pollution Research.

[3]  Muhammed Sit,et al.  D-SRGAN: DEM Super-Resolution with Generative Adversarial Networks , 2020, SN Computer Science.

[4]  Yusuf Sermet,et al.  Crowdsourced approaches for stage measurements at ungauged locations using smartphones , 2020, Hydrological Sciences Journal.

[5]  I. Demir,et al.  A serious gaming framework for decision support on hydrological hazards. , 2020, The Science of the total environment.

[6]  Zhongrun Xiang,et al.  Distributed long-term hourly streamflow predictions using deep learning - A case study for State of Iowa , 2020, Environ. Model. Softw..

[7]  Jun Yan,et al.  A Rainfall‐Runoff Model With LSTM‐Based Sequence‐to‐Sequence Learning , 2020, Water Resources Research.

[8]  Muhammed Sit,et al.  Hydrology@Home: a distributed volunteer computing framework for hydrological research and applications , 2019, Journal of Hydroinformatics.

[9]  Ibrahim Demir,et al.  Optimized watershed delineation library for server-side and client-side web applications , 2019, Open Geospatial Data, Software and Standards.

[10]  Marian Muste,et al.  A web-based decision support system for collaborative mitigation of multiple water-related hazards using serious gaming. , 2019, Journal of environmental management.

[11]  Soheil Ghafurian,et al.  Restoration of Marker Occluded Hematoxylin and Eosin Stained Whole Slide Histology Images Using Generative Adversarial Networks , 2019, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[12]  Ibrahim Demir,et al.  An integrated web framework for HAZUS-MH flood loss estimation analysis , 2019, Natural Hazards.

[13]  Ibrahim Demir,et al.  Towards an information centric flood ontology for information management and communication , 2019, Earth Science Informatics.

[14]  Giha Lee,et al.  Application of Long Short-Term Memory (LSTM) Neural Network for Flood Forecasting , 2019, Water.

[15]  Bong-Chul Seo,et al.  A pilot infrastructure for searching rainfall metadata and generating rainfall product using the big data of NEXRAD , 2019, Environ. Model. Softw..

[16]  Ibrahim Demir,et al.  Decentralized Flood Forecasting Using Deep Neural Networks , 2019, ArXiv.

[17]  Ibrahim Demir,et al.  Identifying disaster-related tweets and their semantic, spatial and temporal context using deep learning, natural language processing and spatial analysis: a case study of Hurricane Irma , 2019, Int. J. Digit. Earth.

[18]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Yang Hong,et al.  Exploring Deep Neural Networks to Retrieve Rain and Snow in High Latitudes Using Multisensor and Reanalysis Data , 2018, Water Resources Research.

[20]  Ibrahim Demir,et al.  An intelligent system on knowledge generation and communication about flooding , 2018, Environ. Model. Softw..

[21]  Cordelia Schmid,et al.  How good is my GAN? , 2018, ECCV.

[22]  Ibrahim Demir,et al.  FLOODSS: Iowa flood information system as a generalized flood cyberinfrastructure , 2018 .

[23]  W. Krajewski,et al.  The Iowa Watersheds Project: Iowa's prototype for engaging communities and professionals in watershed hazard mitigation , 2018 .

[24]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[25]  Alexander G. Schwing,et al.  Generative Modeling Using the Sliced Wasserstein Distance , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[27]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[28]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[29]  Chunhua Shen,et al.  Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition , 2017 .

[30]  Ibrahim Demir,et al.  Optimization of river network representation data models for web‐based systems , 2017 .

[31]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[32]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Kiyoshi Tanaka,et al.  ArtGAN: Artwork synthesis with conditional categorical GANs , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[34]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[35]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[37]  Zhe Gan,et al.  Variational Autoencoder for Deep Learning of Images, Labels and Captions , 2016, NIPS.

[38]  Dawei Han,et al.  Big data and hydroinformatics , 2016 .

[39]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[40]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[42]  Yinda Zhang,et al.  LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop , 2015, ArXiv.

[43]  I. Yucel,et al.  Calibration and evaluation of a flood forecasting system: Utility of numerical weather prediction model, data assimilation and satellite-based rainfall , 2015 .

[44]  C. Gleason,et al.  Retrieval of river discharge solely from satellite imagery and at‐many‐stations hydraulic geometry: Sensitivity to river form and optimization parameters , 2014 .

[45]  Tamlin M. Pavelsky,et al.  Using width‐based rating curves from spatially discontinuous satellite imagery to monitor river discharge , 2014 .

[46]  Jonathan Krause,et al.  3D Object Representations for Fine-Grained Categorization , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[47]  Hanqiang Cao,et al.  Nearest Neighbor Value Interpolation , 2012, ArXiv.

[48]  Geoffrey E. Hinton,et al.  On deep generative models with applications to recognition , 2011, CVPR 2011.

[49]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[50]  J. Hooke.  Complexity, self-organisation and variation in behaviour in meandering rivers , 2007 .

[51]  Robert J. Abrahart,et al.  Hydroinformatics: computational intelligence and technological developments in water science applications—Editorial , 2007 .

[52]  Soo Chin Liew,et al.  The Mekong from satellite imagery: A quick look at a large river , 2007 .

[53]  Michael C. Wendl,et al.  Argonaute—a database for gene regulation by mammalian microRNAs , 2005, BMC Bioinformatics.

[54]  A. Porporato,et al.  On the long‐term behavior of meandering rivers , 2005 .

[55]  Bernard A. Engel,et al.  Web-based GIS and spatial decision support system for watershed management , 2005 .

[56]  Chen Ping,et al.  Evaluation of part of the Mekong River using satellite imagery , 2002 .

[57]  Richard C. Brower,et al.  Geometrical models of interface evolution , 1984 .

[58]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[59]  Adversarial Networks , 2021, Computer Vision.

[60]  Alexandre Alahi,et al.  Pedestrian Image Generation for Self-driving Cars , 2019 .

[61]  Jinsung Yoon,et al.  GENERATIVE ADVERSARIAL NETS , 2018 .

[62]  Rina Tse,et al.  Towards general semi-supervised clustering using a cognitive reinforcement K-Iteration fast learning artificial neural network (R-Kflann). , 2010 .