Importance of bitwise identical reproducibility in earth system modeling and status report

Abstract. Reproducibility is a fundamental principle of scientific research. Bitwise identical reproducibility, i.e., bitwise computational results can be reproduced, guarantees the reproduction of exactly the same results. Here we show the importance of bitwise identical reproducibility to Earth system modeling but the importance has not yet been widely recognized. Modeled mean climate states, variability and trends at different scales may be significantly changed or even lead to opposing results due to a slight change in the original simulation setting during a reproduction. Out of the large body of Earth system modeling publications, few thoroughly describe the whole original simulation setting. As a result, the reproduction of a particular simulation experiment by fellow scientists heavily depends on the interaction with the original authors, which is often inconvenient or even impossible. We anticipate bitwise identical reproducibility to be promoted as a worldwide standard, to guarantee the independent reproduction of simulation results and to further improve model development and scientific research.

[1]  Bin Wang,et al.  The flexible global ocean-atmosphere-land system model, Grid-point Version 2: FGOALS-g2 , 2013, Advances in Atmospheric Sciences.

[2]  Darrel C. Ince,et al.  The case for open computer programs , 2012, Nature.

[3]  Marie-Pierre Moine,et al.  Development and exploitation of a controlled vocabulary in support of climate modelling , 2014 .

[4]  Julia C. Hargreaves,et al.  Editorial: The publication of geoscientific model developments v1.0 , 2013 .

[5]  Sarah Callaghan,et al.  Documenting Climate Models and Their Simulations , 2013 .

[6]  Cecelia DeLuca,et al.  Describing Earth system simulations with the Metafor CIM , 2012 .

[7]  Steve M. Easterbrook,et al.  Engineering the Software for Understanding Climate Change , 2009, Computing in Science & Engineering.

[8]  Bruce Alberts,et al.  Making Data Maximally Available , 2011, Science.

[9]  G. Danabasoglu,et al.  The Community Climate System Model Version 4 , 2011 .

[10]  Peter J. Webster,et al.  Monsoon and Enso: Selectively Interactive Systems , 1992 .

[11]  Fangli Qiao,et al.  Influence of parallel computational uncertainty on simulations of the Coupled General Climate Model , 2011 .

[12]  Bin Wang,et al.  Enhancement for bitwise identical reproducibility of Earth system modeling on the C-Coupler platform , 2015 .

[13]  Ji-Hoon Kang,et al.  An Evaluation of the Software System Dependency of a Global Atmospheric Model , 2013 .

[14]  Steve Easterbrook,et al.  Open code for open science , 2014 .