Quality Assurance and Error Identification for the Community Earth System Model

Earth system models are valuable tools for furthering our understanding of past, present, and future climate states. Because these models tend to be large and complex as well as in a state of near constant development, quality assurance (and subsequent debugging) are critical pieces in the development cycle. Here, we describe our multi-year effort to better evaluate the quality and "correctness" of the Community Earth System Model (CESM), a widely-used climate model. Our approach depends on an initial coarse-grain ensemble-based consistency test to determine code correctness, which has already proved successful in practice. The additional capability desired is a means of easily tracing a coarse-grain failure to its root cause, and we discuss our strategy and promising efforts to date toward that goal.

[1]  Daniel John Milroy Refining the composition of CESM-ECT ensembles , 2015 .

[2]  Elizabeth R. Jessup,et al.  Nine time steps: ultra-fast statistical consistency testing of the Community Earth System Model (pyCECT v3.0) , 2017 .

[3]  John M. Dennis,et al.  KGEN: A Python Tool for Automated Fortran Kernel Generation and Verification , 2016, ICCS.

[4]  Sheri Mickelson,et al.  A new ensemble-based consistency test for the Community Earth System Model (pyCECT v1.0) , 2015 .

[5]  Tso-Jung Yen,et al.  Discussion on "Stability Selection" by Meinshausen and Buhlmann , 2010 .

[6]  A. Meijers,et al.  The Southern Ocean in the Coupled Model Intercomparison Project phase 5 , 2014, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[7]  W. Washington,et al.  An Introduction to Three-Dimensional Climate Modeling , 1986 .

[8]  Jon Pipitone,et al.  Assessing climate model software quality: a defect density analysis of three models , 2012 .

[9]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[10]  K.,et al.  The Community Earth System Model (CESM) large ensemble project: a community resource for studying climate change in the presence of internal climate variability , 2015 .

[11]  N. Meinshausen,et al.  Stability selection , 2008, 0809.2932.

[12]  W. Collins,et al.  The Community Earth System Model: A Framework for Collaborative Research , 2013 .

[13]  David L. Williamson,et al.  The Accumulation of Rounding Errors and Port Validation for Global Atmospheric Models , 1997, SIAM J. Sci. Comput..

[14]  Steve M. Easterbrook,et al.  Guest Editors' Introduction: Climate Change - Science and Software , 2011, IEEE Software.

[15]  Tom M. L. Wigley,et al.  Ensemble Simulation of Twenty-First Century Climate Changes: Business-as-Usual versus CO2 Stabilization , 2001 .

[16]  Douglas W. Nychka,et al.  A new ensemble-based consistency test for the Community Earth System Model , 2015 .

[17]  Dorit Hammerling,et al.  Towards Characterizing the Variability of Statistically Consistent Community Earth System Model Simulations , 2016, ICCS.

[18]  Guangwen Yang,et al.  Evaluating statistical consistency in the ocean model component of the Community Earth System Model (pyCECT v2.0) , 2016 .

[19]  Lawrence Buja,et al.  The computational future for climate and Earth system models: on the path to petaflop and beyond , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.