Deep learning to represent subgrid processes in climate models

Significance Current climate models are too coarse to resolve many of the atmosphere’s most important processes. Traditionally, these subgrid processes are heuristically approximated in so-called parameterizations. However, imperfections in these parameterizations, especially for clouds, have impeded progress toward more accurate climate predictions for decades. Cloud-resolving models alleviate many of the gravest issues of their coarse counterparts but will remain too computationally demanding for climate change predictions for the foreseeable future. Here we use deep learning to leverage the power of short-term cloud-resolving simulations for climate modeling. Our data-driven model is fast and accurate, thereby showing the potential of machine-learning–based approaches to climate model development. The representation of nonlinear subgrid processes, especially clouds, has been a major source of uncertainty in climate models for decades. Cloud-resolving models better represent many of these processes and can now be run globally but only for short-term simulations of at most a few years because of computational limitations. Here we demonstrate that deep learning can be used to capture many advantages of cloud-resolving modeling at a fraction of the computational cost. We train a deep neural network to represent all atmospheric subgrid processes in a climate model by learning from a multiscale model in which convection is treated explicitly. The trained neural network then replaces the traditional subgrid parameterizations in a global general circulation model in which it freely interacts with the resolved dynamics and the surface-flux scheme. The prognostic multiyear simulations are stable and closely reproduce not only the mean climate of the cloud-resolving simulation but also key aspects of variability, including precipitation extremes and the equatorial wave spectrum. Furthermore, the neural network approximately conserves energy despite not being explicitly instructed to. Finally, we show that the neural network parameterization generalizes to new surface forcing patterns but struggles to cope with temperatures far outside its training manifold. Our results show the feasibility of using deep learning for climate model parameterization. In a broader context, we anticipate that data-driven Earth system model development could play a key role in reducing climate prediction uncertainty in the coming decade.

[1]  H. Yashiro,et al.  Deep moist atmospheric convection in a subkilometer global simulation , 2013 .

[2]  Matthew C. Wheeler,et al.  Convectively Coupled Equatorial Waves: Analysis of Clouds and Temperature in the Wavenumber–Frequency Domain , 1999 .

[3]  W. Skamarock,et al.  The resolution dependence of explicitly modeled convective systems , 1997 .

[4]  Noah D. Brenowitz,et al.  Prognostic Validation of a Neural Network Unified Physics Parameterization , 2018, Geophysical Research Letters.

[5]  Tsuyoshi Yamaura,et al.  Resolution Dependence of the Diurnal Cycle of Precipitation Simulated by a Global Cloud-System Resolving Model , 2016 .

[6]  Pierre Gentine,et al.  A Probabilistic Bulk Model of Coupled Mixed Layer and Convection. Part I: Clear-Sky Case , 2013 .

[7]  Pierre Gentine,et al.  Could Machine Learning Break the Convection Parameterization Deadlock? , 2018, Geophysical Research Letters.

[8]  Vladimir M. Krasnopolsky,et al.  Using Ensemble of Neural Networks to Learn Stochastic Convection Parameterizations for Climate and Numerical Weather Prediction Models from Data Simulated by a Cloud Resolving Model , 2013, Adv. Artif. Neural Syst..

[9]  Sandrine Mouysset,et al.  Experiences in multiyear combined state-parameter estimation with an ecosystem model of the North Atlantic and Arctic Oceans using the Ensemble Kalman Filter , 2015 .

[10]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[13]  G. Vecchi,et al.  The vertical distribution of cloud feedback in coupled ocean‐atmosphere models , 2011 .

[14]  P. O'Gorman,et al.  Using Machine Learning to Parameterize Moist Convection: Potential for Modeling of Climate, Climate Change, and Extreme Events , 2018, Journal of Advances in Modeling Earth Systems.

[15]  A. P. Siebesma,et al.  Clouds, circulation and climate sensitivity , 2015 .

[16]  Jian Sun,et al.  Effects of explicit convection on global land‐atmosphere coupling in the superparameterized CAM , 2015 .

[17]  D. Lüthi,et al.  Evaluation of the convection‐resolving climate modeling approach on continental scales , 2017 .

[18]  Andrew Stuart,et al.  Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High‐Resolution Simulations , 2017, 1709.00037.

[19]  A. P. Siebesma,et al.  Climate goals and computing the future of clouds , 2017 .

[20]  Sandrine Bony,et al.  What favors convective aggregation and why? , 2015 .

[21]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[22]  Jean-Philippe Lafore,et al.  A Density Current Parameterization Coupled with Emanuel’s Convection Scheme. Part I: The Models , 2010 .

[23]  Fuqing Zhang,et al.  Ensemble‐based simultaneous state and parameter estimation with MM5 , 2006 .

[24]  S. Bony,et al.  What Are Climate Models Missing? , 2013, Science.

[25]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[26]  Richard Neale,et al.  Parameterizing Convective Organization to Escape the Entrainment Dilemma , 2011 .

[27]  David A. Randall,et al.  Impacts of cloud superparameterization on projected daily rainfall intensity climate changes in multiple versions of the Community Earth System Model , 2016 .

[28]  Hartwig Deneke,et al.  Large‐eddy simulations over Germany using ICON: a comprehensive evaluation , 2017 .

[29]  Abhinav Vishnu,et al.  Deep learning for computational chemistry , 2017, J. Comput. Chem..

[30]  Pierre Gentine,et al.  Representation of daytime moist convection over the semi‐arid Tropics by parametrizations used in climate and meteorological models , 2015 .

[31]  Chidong Zhang,et al.  Madden‐Julian Oscillation , 2005 .

[32]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[33]  A. Arakawa The Cumulus Parameterization Problem: Past, Present, and Future , 2004 .

[34]  David A. Randall,et al.  Global‐scale convective aggregation: Implications for the Madden‐Julian Oscillation , 2015 .

[35]  S. Bony,et al.  Spread in model climate sensitivity traced to atmospheric convective mixing , 2014, Nature.

[36]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[37]  David A. Randall,et al.  Structure of the Madden-Julian Oscillation in the Superparameterized CAM , 2009 .

[38]  Paul Ginoux,et al.  CLUBB as a unified cloud parameterization: Opportunities and challenges , 2015 .

[39]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[40]  Andrew Gettelman,et al.  The Art and Science of Climate Model Tuning , 2017 .

[41]  M. Pritchard,et al.  Rainfall From Resolved Rather Than Parameterized Processes Better Represents the Present‐Day and Climate Change Response of Moderate Rates in the Community Atmosphere Model , 2018, Journal of advances in modeling earth systems.

[42]  N. McFarlane,et al.  Sensitivity of Climate Simulations to the Parameterization of Cumulus Convection in the Canadian Climate Centre General Circulation Model , 1995, Data, Models and Analysis.

[43]  W. Collins,et al.  The Formulation and Atmospheric Simulation of the Community Atmosphere Model Version 3 (CAM3) , 2006 .

[44]  D. Randall,et al.  A cloud resolving model as a cloud parameterization in the NCAR Community Climate System Model: Preliminary results , 2001 .

[45]  P. Baldi,et al.  Searching for exotic particles in high-energy physics with deep learning , 2014, Nature Communications.

[46]  C. Bretherton,et al.  The Soil Moisture–Precipitation Feedback in Simulations with Explicit and Parameterized Convection , 2009 .

[47]  Xianan Jiang,et al.  Key processes for the eastward propagation of the Madden‐Julian Oscillation based on multimodel simulations , 2017 .

[48]  C. Bretherton,et al.  Restricting 32–128 km horizontal scales hardly affects the MJO in the Superparameterized Community Atmosphere Model v.3.0 but the number of cloud‐resolving grid columns constrains vertical mixing , 2014 .

[49]  Stefan N. Tulich,et al.  A strategy for representing the effects of convective momentum transport in multiscale models: Evaluation using a new superparameterized version of the Weather Research and Forecast model (SP‐WRF) , 2015 .

[50]  Daniel Klocke,et al.  Rediscovery of the doldrums in storm-resolving simulations over the tropical Atlantic , 2017, Nature Geoscience.

[51]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[52]  S. Bony,et al.  Marine boundary layer clouds at the heart of tropical cloud feedback uncertainties in climate models , 2005 .

[53]  C. Bretherton,et al.  Toward low‐cloud‐permitting cloud superparameterization with explicit boundary layer turbulence , 2017 .

[54]  Yen-Ting Hwang,et al.  Link between the double-Intertropical Convergence Zone problem and cloud biases over the Southern Ocean , 2013, Proceedings of the National Academy of Sciences.

[55]  G. Bellon,et al.  The double ITCZ bias in CMIP5 models: interaction between SST, large-scale circulation and precipitation , 2015, Climate Dynamics.

[56]  Christopher S. Bretherton,et al.  Sensitivity of Coupled Tropical Pacific Model Biases to Convective Parameterization in CESM1 , 2018 .

[57]  Z. Kuang,et al.  Moist Static Energy Budget of MJO-like Disturbances in the Atmosphere of a Zonally Symmetric Aquaplanet , 2012 .

[58]  C. Bretherton,et al.  Convective self‐aggregation feedbacks in near‐global cloud‐resolving simulations of an aquaplanet , 2015 .

[59]  Joao Teixeira,et al.  A Unified Model for Moist Convective Boundary Layers Based on a Stochastic Eddy-Diffusivity/Mass-Flux Parameterization , 2013 .

[60]  Peter Bauer,et al.  The quiet revolution of numerical weather prediction , 2015, Nature.

[61]  Peter A. Bogenschutz,et al.  Simulation, Modeling, and Dynamically Based Parameterization of Organized Tropical Convection for Global Climate Models , 2017 .