Bitwise identical compiling setup: prospective for reproducibility and reliability of earth system modeling

Abstract. Reproducibility and reliability are fundamental principles of scientific research. A compiling setup that includes a specific compiler version and compiler flags is an essential technical support for Earth system modeling. With the fast development of computer software and hardware, a compiling setup has to be updated frequently, which challenges the reproducibility and reliability of Earth system modeling. The existing results of a simulation using an original compiling setup may be irreproducible by a newer compiling setup because trivial round-off errors introduced by the change in compiling setup can potentially trigger significant changes in simulation results. Regarding the reliability, a compiler with millions of lines of code may have bugs that are easily overlooked due to the uncertainties or unknowns in Earth system modeling. To address these challenges, this study shows that different compiling setups can achieve exactly the same (bitwise identical) results in Earth system modeling, and a set of bitwise identical compiling setups of a model can be used across different compiler versions and different compiler flags. As a result, the original results can be more easily reproduced; for example, the original results with an older compiler version can be reproduced exactly with a newer compiler version. Moreover, this study shows that new test cases can be generated based on the differences of bitwise identical compiling setups between different models, which can help detect software bugs in the codes of models and compilers and finally improve the reliability of Earth system modeling.

[1]  Ji-Hoon Kang,et al.  An Evaluation of the Software System Dependency of a Global Atmospheric Model , 2013 .

[2]  W. Collins,et al.  The Community Earth System Model: A Framework for Collaborative Research , 2013 .

[3]  Steve Easterbrook,et al.  The software architecture of climate models: a graphical comparison of CMIP5 and EMICAR5 configurations , 2015 .

[4]  W. Collins,et al.  Description of the NCAR Community Atmosphere Model (CAM 3.0) , 2004 .

[5]  Mariana Vertenstein,et al.  The Parallel Ocean Program (POP) reference manual: Ocean component of the Community Climate System Model (CCSM) , 2010 .

[6]  Steve M. Easterbrook,et al.  Engineering the Software for Understanding Climate Change , 2009, Computing in Science & Engineering.

[7]  Peter E. Thornton,et al.  Technical Description of the Community Land Model (CLM) , 2004 .

[8]  Z. Xuehong,et al.  An eddy-permitting oceanic general circulation model and its preliminary evaluation , 2004 .

[9]  Guangwen Yang,et al.  Evaluation of grid-point atmospheric model of IAP LASG version 2 (GAMIL2) , 2013, Advances in Atmospheric Sciences.

[10]  Jiping Liu,et al.  Sensitivity of sea ice and ocean simulations to sea ice salinity in a coupled global climate model , 2010 .

[11]  Bin Wang,et al.  The flexible global ocean-atmosphere-land system model, Grid-point Version 2: FGOALS-g2 , 2013, Advances in Atmospheric Sciences.

[12]  Douglas W. Nychka,et al.  A new ensemble-based consistency test for the Community Earth System Model , 2015 .

[13]  Geng Yang,et al.  Importance of bitwise identical reproducibility in earth system modeling and status report , 2015 .

[14]  Fangli Qiao,et al.  Influence of parallel computational uncertainty on simulations of the Coupled General Climate Model , 2011 .

[15]  Bin Wang,et al.  Enhancement for bitwise identical reproducibility of Earth system modeling on the C-Coupler platform , 2015 .