Software Testing and Verification in Climate Model Development

Over the past 30 years, most climate models have grown from relatively simple representations of a few atmospheric processes to complex multidisciplinary systems. Computer infrastructure over that period has gone from punchcard mainframes to modern parallel clusters. Model implementations have become complex, brittle, and increasingly difficult to extend and maintain. Verification processes for model implementations rely almost exclusively on some combination of detailed analyses of output from full climate simulations and system-level regression tests. Besides being costly in terms of developer time and computing resources, these testing methodologies are limited in the types of defects they can detect, isolate, and diagnose. Mitigating these weaknesses of coarse-grained testing with finer-grained unit tests has been perceived as cumbersome and counterproductive. Recent advances in commercial software tools and methodologies have led to a renaissance of systematic fine-grained testing. This opens new possibilities for testing climate-modeling-software methodologies.

[1]  Cecelia DeLuca,et al.  The architecture of the Earth System Modeling Framework , 2003, Computing in Science & Engineering.

[2]  V. Canuto,et al.  Present-Day Atmospheric Simulations Using GISS ModelE: Comparison to In Situ, Satellite, and Reanalysis Data , 2006 .

[3]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[4]  Michael C. Feathers.  Working Effectively with Legacy Code , 2004, XP/Agile Universe.

[5]  Douglass E. Post,et al.  Software Project Management and Quality Engineering Practices for Complex, Coupled Multiphysics, Massively Parallel Computational Simulations: Lessons Learned From ASCI , 2004, Int. J. High Perform. Comput. Appl..

[6]  Gregory Tassey,et al.  Prepared for what , 2007 .

[7]  D. Griffeath,et al.  Modeling snow-crystal growth: a three-dimensional mesoscopic approach. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[9]  Hélène Guillemot,et al.  Connections between simulations and observation in climate computer modeling. Scientist’s practices and “bottom-up epistemology” lessons , 2010 .

[10]  Steve M. Easterbrook,et al.  Engineering the Software for Understanding Climate Change , 2009, Computing in Science & Engineering.

[11]  G. Visconti,et al.  Diagnostic Tools in Atmospheric Physics , 1995 .

[12]  L. Alamos,et al.  Massively Parallel Computational Simulations : Lessons Learned From ASCI Software Project Management and Quality Engineering Practices for Complex , Coupled Multiphysics , 2022 .