What is Validation of Computer Simulations? Toward a Clarification of the Concept of Validation and of Related Notions

This chapter clarifies the concept of validation of computer simulations by comparing various definitions that have been proposed for the notion. While the definitions agree in taking validation to be an evaluation, they differ on the following questions: (1) What exactly is evaluated—results from a computer simulation, a model, a computer code? (2) What are the standards of evaluation––truth, accuracy, and credibility or also something else? (3) What type of verdict does validation lead to––that the simulation is such and such good, or that it passes a test defined by a certain threshold? (4) How strong needs the case to be for the verdict? (5) Does validation necessarily proceed by comparing simulation outputs with measured data? Along with these questions, the chapter explains notions that figure prominently in them, e.g., the concepts of accuracy and credibility. It further discusses natural answers to the questions as well as arguments that speak in favor and against these answers. The aim is to obtain a better understanding of the options we have for defining validation and how they are related to each other.

[1]  W. Parker.  II—Wendy S. Parker: Confirmation and adequacy-for-Purpose in Climate Modelling , 2009 .

[2]  N Oreskes,et al.  Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences , 1994, Science.

[3]  Thomas H. Naylor,et al.  Verification of Computer Simulation Models , 1967 .

[4]  Timothy G. Trucano,et al.  Verification and validation benchmarks , 2008 .

[5]  James Bown,et al.  Argument-Driven Validation of Computer Simulations - A Necessity, Rather than an Option , 2010, 2010 Second International Conference on Advances in System Testing and Validation Lifecycle.

[6]  Carl G. Hempel,et al.  I.—STUDIES IN THE LOGIC OF CONFIRMATION (II.) , 1945 .

[7]  G. Harman.  The Inference to the Best Explanation , 1965 .

[8]  Stephan Hartmann,et al.  The World as a Process: Simulations in the Natural and Social Sciences , 1996 .

[9]  J. Woodward,et al.  Saving the phenomena , 1988 .

[10]  Bas C. van Fraassen,et al.  The Scientific Image , 1980 .

[11]  David B. Resnik,et al.  Is the precautionary principle unscientific , 2003 .

[12]  Christopher J. Roy,et al.  Verification and Validation in Scientific Computing , 2010 .

[13]  Claus Beisbart,et al.  What Is Understanding? An Overview of Recent Debates in Epistemology and Philosophy of Science , 2017 .

[14]  G. Brun.  Explication as a Method of Conceptual Re-engineering , 2016 .

[15]  Wolfgang Künne,et al.  Conceptions of truth , 2003 .

[16]  H. Cappelen.  Fixing Language. An Essay on Conceptual Engineering , 2018, Zeitschrift für philosophische Forschung.

[17]  Gunnar Abrahamson,et al.  Terminology for model credibility , 1980 .

[18]  Daniela M. Bailer-Jones,et al.  When scientific models represent , 2003 .

[19]  Patrick J. Roache,et al.  Verification and Validation in Computational Science and Engineering , 1998 .

[20]  Wendy S. Parker,et al.  Franklin, Holmes, and the Epistemology of Computer Simulation , 2008 .

[21]  Paul Humphreys,et al.  Extending Ourselves: Computational Science, Empiricism, and Scientific Method , 2004 .

[22]  Mauricio Suárez,et al.  An Inferential Conception of Scientific Representation , 2004, Philosophy of Science.

[23]  Lefteris Farmakis.  Inference to the Best Explanation, 2nd edition , 2004 .

[24]  Gertrude Hirsch Hadorn,et al.  Building confidence in climate model projections: an analysis of inferences from fit , 2017 .

[25]  H. Lacey.  Is Science Value Free?: Values and Scientific Understanding , 2004 .

[26]  Claus Beisbart,et al.  How can computer simulations produce new knowledge? , 2012 .

[27]  Eric Winsberg,et al.  Sanctioning Models: The Epistemology of Simulation , 1999, Science in Context.

[28]  J. Norton.  A Material Theory of Induction* , 2003, Philosophy of Science.

[29]  Patrick J. Roache,et al.  Perspective: Validation—What Does It Mean? , 2009 .

[30]  Claus Beisbart.  Advancing Knowledge Through Computer Simulations? A Socratic Exercise , 2017 .

[31]  Till Grüne-Yanoff,et al.  The explanatory potential of artificial societies , 2009, Synthese.

[32]  Claus Beisbart,et al.  Are We Sims? How Computer Simulations Represent and What this Means for the Simulation Argument , 2014 .

[33]  Leonard E. Schwer,et al.  An overview of the PTC 60/V&V 10: guide for verification and validation in computational solid mechanics , 2007, Engineering with Computers.