Automated continuous verification for numerical simulation

Abstract. Verification is a process crucially important for the final users of a computational model: code is useless if its results cannot be relied upon. Typically, verification is seen as a discrete event, performed once and for all after development is complete. However, this does not reflect the reality that many geoscientific codes undergo continuous development of the mathematical model, discretisation and software implementation. Therefore, we advocate that in such cases verification must be continuous and happen in parallel with development: the desirability of their automation follows immediately. This paper discusses a framework for automated continuous verification of wide applicability to any kind of numerical simulation. It also documents a range of test cases to show the possibilities of the framework.

[1]  C.R.E. de Oliveira,et al.  Three-dimensional unstructured mesh ocean modelling , 2005 .

[2]  Eckart Meiburg,et al.  Analysis and direct numerical simulation of the flow at a gravity-current head. Part 1. Flow topology and front speed for slip and no-slip boundaries , 2000, Journal of Fluid Mechanics.

[3]  P. Swarztrauber,et al.  A standard test set for numerical approximations to the shallow water equations in spherical geometry , 1992 .

[4]  O. Botella,et al.  BENCHMARK SPECTRAL RESULTS ON THE LID-DRIVEN CAVITY FLOW , 1998 .

[6]  C. Bruneau,et al.  The 2D lid-driven cavity problem revisited , 2006 .

[7]  C C Pain,et al.  Anisotropic mesh adaptivity for multi-scale ocean modelling , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[8]  E. Erturk,et al.  Numerical solutions of 2‐D steady incompressible driven cavity flow at high Reynolds numbers , 2004, ArXiv.

[9]  John C. Cherniavsky,et al.  Validation, Verification, and Testing of Computer Software , 1982, CSUR.

[10]  J. Tinsley Oden,et al.  Verification and validation in computational engineering and science: basic concepts , 2004 .

[11]  L. Perelman,et al.  A finite-volume, incompressible Navier Stokes model for studies of the ocean on parallel computers , 1997 .

[12]  Steve M. Easterbrook,et al.  Engineering the Software for Understanding Climate Change , 2009, Computing in Science & Engineering.

[13]  V. Legat,et al.  A three-dimensional unstructured mesh finite element shallow-water model, with application to the flows around an island and in a wind-driven, elongated basin , 2008 .

[14]  P. Delecluse,et al.  OPA 8.1 Ocean General Circulation Model reference manual , 1998 .

[15]  A. Staniforth,et al.  A new dynamical core for the Met Office's global and regional modelling of the atmosphere , 2005 .

[16]  Timothy G. Trucano,et al.  Verification and validation. , 2005 .

[17]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[18]  Matthew D. Piggott,et al.  The impact of mesh adaptivity on the gravity current front speed in a two-dimensional lock-exchange , 2011 .

[19]  P. P. Brown,et al.  Sphere Drag and Settling Velocity Revisited , 2003 .

[20]  Susan L. Graham,et al.  Gprof: A call graph execution profiler , 1982, SIGPLAN '82.

[21]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[22]  Lloyd N. Trefethen.  IV.21 Numerical Analysis , 2010 .

[23]  William E. Howden,et al.  Reliability of the Path Analysis Testing Strategy , 1976, IEEE Transactions on Software Engineering.

[24]  Beth A. Wingate,et al.  A better, more discriminating test problem for ocean tracer transport , 2000 .

[25]  Francis X. Giraldo,et al.  A study of spectral element and discontinuous Galerkin methods for the Navier-Stokes equations in nonhydrostatic mesoscale atmospheric modeling: Equation sets and test cases , 2008, J. Comput. Phys..

[26]  Timothy G. Trucano,et al.  Verification and Validation in Computational Fluid Dynamics , 2002 .

[27]  Colin J. Cotter,et al.  Spud 1.0: generalising and automating the user interfaces of scientific computer models , 2008 .

[28]  Michael Pilato.  Version Control with Subversion , 2004 .

[29]  William Schroeder,et al.  The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics , 1997 .

[30]  M. Gerritsen,et al.  An unstructured-grid, finite-volume, nonhydrostatic, parallel coastal ocean simulator , 2006 .

[31]  Patrick Knupp,et al.  Code Verification by the Method of Manufactured Solutions , 2000 .

[32]  Christopher C. Pain,et al.  A new computational framework for multi‐scale ocean modelling based on adapting unstructured meshes , 2008 .

[33]  Dale B. Haidvogel,et al.  Numerical Ocean Circulation Modeling , 1999 .

[34]  Curtis C. Ober,et al.  Measuring progress in order-verification within software development projects , 2007, Engineering with Computers.

[35]  Michael Manga,et al.  Gravity Currents in the Environment and the Laboratory, by John E. Simpson , 1999 .

[36]  D. Haidvogel,et al.  A semi-implicit ocean circulation model using a generalized topography-following coordinate system , 1994 .

[37]  H. Stommel,et al.  The westward intensification of wind‐driven ocean currents , 1948 .

[38]  C.R.E. de Oliveira,et al.  Tetrahedral mesh optimisation and adaptivity for steady-state and transient finite element calculations , 2001 .

[39]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[40]  Hugh W. Coleman,et al.  Comprehensive Approach to Verification and Validation of CFD Simulations—Part 1: Methodology and Procedures , 2001 .

[41]  P. Roache.  Code Verification by the Method of Manufactured Solutions , 2002 .