ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods

Climate change is a major threat to humanity, and the actions required to prevent its catastrophic consequences include changes in both policy-making and individual behaviour. However, taking action requires understanding the effects of climate change, even though they may seem abstract and distant. Projecting the potential consequences of extreme climate events such as flooding in familiar places can help make the abstract impacts of climate change more concrete and encourage action. As part of a larger initiative to build a website that projects extreme climate events onto user-chosen photos, we present our solution to simulate photo-realistic floods on authentic images. To address this complex task in the absence of suitable training data, we propose ClimateGAN, a model that leverages both simulated and real data for unsupervised domain adaptation and conditional image generation. In this paper, we describe the details of our framework, thoroughly evaluate components of our architecture and demonstrate that our model is capable of robustly generating photo-realistic flooding.

[1]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[3]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[5]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Peter Wonka,et al.  SEAN: Image Synthesis With Semantic Region-Adaptive Normalization , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Konrad Schindler,et al.  Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Jaakko Lehtinen,et al.  Few-Shot Unsupervised Image-to-Image Translation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[9]  B. Bishop,et al.  Special issue article: The social psychology of climate change Imagining climate change: The role of implicit associations and affective psychological distancing in climate change responses , 2014 .

[10]  Iasonas Kokkinos,et al.  Attentive Single-Tasking of Multiple Tasks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Björn-Ola Linnér,et al.  Climate change effects at your doorstep: Geographic visualization to support Nordic homeowners in adapting to climate change , 2016 .

[12]  Björn-Ola Linnér,et al.  Visualization for supporting individual climate change adaptation planning: Assessment of a web-based tool , 2017 .

[13]  Andreas Dengel,et al.  Adversarial Text-to-Image Synthesis: A Review , 2021, Neural Networks.

[14]  J. Berenguer,et al.  The Effect of Empathy in Proenvironmental Attitudes and Behaviors , 2007 .

[15]  Jeremy N. Bailenson,et al.  Short- and long-term effects of embodied experiences in immersive virtual environments on environmental locus of control and behavior , 2014, Comput. Hum. Behav..

[16]  R. Nicholls,et al.  Future Coastal Population Growth and Exposure to Sea-Level Rise and Coastal Flooding - A Global Assessment , 2015, PloS one.

[17]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[18]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[19]  L. Feyen,et al.  Development and evaluation of a framework for global flood hazard mapping , 2016 .

[20]  Adam Corner,et al.  Climate visuals: A mixed methods investigation of public perceptions of climate images in three countries , 2016 .

[21]  Alexandre Lacoste,et al.  Quantifying the Carbon Emissions of Machine Learning , 2019, ArXiv.

[22]  Michael S. Bernstein,et al.  HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models , 2019, NeurIPS.

[23]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  R. Glenn Cummins,et al.  Communicating Local Climate Risks Online Through an Interactive Data Visualization , 2017 .

[25]  Xuelong Li,et al.  Weather GAN: Multi-Domain Weather Translation Using Generative Adversarial Networks , 2021, ArXiv.

[26]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[27]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[28]  Tannecia S. Stephenson,et al.  Chapter 3: Impacts of 1.5ºC global warming on natural and human systems , 2018 .

[29]  Seunghoon Hong,et al.  Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[31]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.

[32]  Roberto Cipolla,et al.  Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Andrew J. Davison,et al.  End-To-End Multi-Task Learning With Attention , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Amir H. Behzadan,et al.  Flood depth mapping in street photos with image processing and deep neural networks , 2021, Comput. Environ. Urban Syst..

[35]  Michael Crawshaw,et al.  Multi-Task Learning with Deep Neural Networks: A Survey , 2020, ArXiv.

[36]  Mathew E. Hauer,et al.  Migration induced by sea-level rise could reshape the US population landscape , 2017 .

[37]  Peter Kontschieder,et al.  The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  S. Levine,et al.  Gradient Surgery for Multi-Task Learning , 2020, NeurIPS.