How simulations fail

Abstract‘The problem with simulations is that they are doomed to succeed.’ So runs a common criticism of simulations—that they can be used to ‘prove’ anything and are thus of little or no scientific value. While this particular objection represents a minority view, especially among those who work with simulations in a scientific context, it raises a difficult question: what standards should we use to differentiate a simulation that fails from one that succeeds? In this paper we build on a structural analysis of simulation developed in previous work to provide an evaluative account of the variety of ways in which simulations do fail. We expand the structural analysis in terms of the relationship between a simulation and its real-world target emphasizing the important role of aspects intended to correspond and also those specifically intended not to correspond to reality. The result is an outline both of the ways in which simulations can fail and the scientific importance of those various forms of failure.

[1]  Mark S. Granovetter.  Threshold Models of Collective Behavior , 1978, American Journal of Sociology.

[2]  William F. Bottke,et al.  An asteroid breakup 160 Myr ago as the probable source of the K/T impactor , 2007, Nature.

[3]  M. Nowak,et al.  THE SPATIAL DILEMMAS OF EVOLUTION , 1993 .

[4]  M. Nowak,et al.  Tit for tat in heterogeneous populations , 1992, Nature.

[5]  P. Grim.  The greater generosity of the spatialized prisoner’s dilemma , 1995 .

[6]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[7]  Keith A. Markus,et al.  Making Things Happen: A Theory of Causal Explanation , 2007 .

[8]  M. Suárez,et al.  Scientific representation: against similarity and isomorphism , 2003 .

[9]  Elizabeth E. Bruch,et al.  SPATIAL INEQUALITY, NEIGHBORHOOD MOBILITY, AND RESIDENTIAL SEGREGATION* , 2001 .

[10]  Lorenzo Magnani,et al.  Model-Based Reasoning: Science, Technology, Values , 2012 .

[11]  Wendy S. Parker,et al.  Does matter really matter? Computer simulations, experiments, and materiality , 2009, Synthese.

[12]  Patrick Grim,et al.  A graphic measure for game-theoretic robustness , 2008, Synthese.

[13]  M. Suárez.  Scientific Representation: Against Similarity and Isomorphism , 2003 .

[14]  Hans Radder,et al.  The Philosophy Of Scientific Experimentation , 2003 .

[15]  Steven French,et al.  Science and Partial Truth , 2003 .

[16]  Richard Scheines,et al.  Discovering Causal Structure: Artificial Intelligence, Philosophy of Science, and Statistical Modeling , 1987 .

[17]  R. Canup,et al.  Simulations of a late lunar-forming impact , 2004 .

[18]  T. Schelling.  Micromotives and Macrobehavior , 1978 .

[19]  Mary S. Morgan,et al.  Experiments without material intervention: Model experiments, virtual experiments and virtually experiments , 2000 .

[20]  S. Hartmann,et al.  Models in Science , 2006 .

[21]  Robert L. Axtell,et al.  Population growth and collapse in a multiagent model of the Kayenta Anasazi in Long House Valley , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Francesco Guala,et al.  Models, Simulations, and Experiments , 2002 .

[23]  B A Huberman,et al.  Evolutionary games and computer simulations. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Cyrille Imbert,et al.  Computer simulations as experiments , 2009, Synthese.

[25]  Noël Carroll,et al.  Mimesis as Make-Beleive@@@Mimesis as Make-Believe: On the Foundations of the Representational Arts. , 1995 .

[26]  G. Nigel Gilbert,et al.  Tools and Techniques for Social Science Simulation [Dagstuhl Seminar, May 5-9, 1997] , 1997, Tools and Techniques for Social Science Simulation.

[27]  Phil Husbands,et al.  Reducing Prejudice: A Spatialized Game-Theoretic Model for the Contact Hypothesis , 2004 .

[28]  Joshua M. Epstein,et al.  Understanding Anasazi culture change through agent-based modeling , 2000 .

[29]  Euel Elliott,et al.  Adaptive agents, intelligence, and emergent human organization: Capturing complexity through agent-based modeling , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Patrick Grim,et al.  Reducing Prejudice : A Spatialized Game-Theoretic Model for the Contact Hypothesis , 2004 .

[31]  P. Grim,et al.  Spatialization and greater generosity in the stochastic Prisoner's Dilemma. , 1996, Bio Systems.

[32]  Eric Winsberg,et al.  A tale of two methods , 2009, Synthese.

[33]  Mitchel Resnick,et al.  Turtles, termites, and traffic jams - explorations in massively parallel microworlds , 1994 .

[34]  Viola Schiaffonati,et al.  Representation and Invariance of Scientific Structures , 2006 .

[35]  Johannes Lenhard,et al.  From Hierarchical to Network-Like Integration: A Revolution of Modeling Style in Computer-Simulation , 2006 .

[36]  P. Grim,et al.  Modeling Prejudice Reduction: Spatialized Game Theory and the Contact Hypothesis , 2005 .

[37]  B. V. Fraassen.  Scientific Representation: Paradoxes of Perspective , 2008 .

[38]  Timothy A. Kohler,et al.  Dynamics in human and primate societies: agent-based modeling of social and spatial processes , 2000 .

[39]  Miguel Patiri,et al.  Fictions in Science: Philosophical Essays on Modeling and Idealization , 2012 .

[40]  C. Allen,et al.  Stanford Encyclopedia of Philosophy , 2011 .

[41]  H. Kyburg,et al.  How the laws of physics lie , 1984 .

[42]  Dario Floreano,et al.  Artificial Life X , 2006 .

[43]  Michael Möhring,et al.  Sensitivity Analysis in the Social Sciences: Problems and Prospects , 1997, Tools and Techniques for Social Science Simulation.

[44]  Joshua M Epstein,et al.  Modeling civil violence: An agent-based computational approach , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Bas C. van Fraassen,et al.  The Scientific Image , 1980 .

[46]  Joshua M. Epstein,et al.  The Evolution of Social Behavior in the Prehistoric American Southwest , 2003, Artificial Life.

[47]  Angelo M. Petroni,et al.  Conventionalism, scientific discovery and the sociology of knowledge* , 1993 .

[48]  D. Cummings,et al.  Strategies for mitigating an influenza pandemic , 2006, Nature.

[49]  Eric Winsberg,et al.  Simulated Experiments: Methodology for a Virtual World , 2003, Philosophy of Science.

[50]  Nancy Cartwright,et al.  Aristotelian natures and the modern experimental method , 1999 .

[51]  P. Teller.  Twilight Of The Perfect Model Model , 2001 .

[52]  P. Kitcher.  The Evolution of Human Altruism , 1993 .

[53]  R. Mare,et al.  Neighborhood Choice and Neighborhood Change1 , 2006, American Journal of Sociology.

[54]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[55]  Patrick Grim,et al.  What kind of science is simulation? , 2007, J. Exp. Theor. Artif. Intell..

[56]  Johannes Lenhard,et al.  Simulation : pragmatic construction of reality , 2006 .

[57]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[58]  Christopher G. Langton,et al.  Life at the Edge of Chaos , 1992 .

[59]  Rachelle D. Hollander,et al.  Acceptable Evidence: Science and Values in Risk Management , 1994 .

[60]  R. Giere.  How Models Are Used to Represent Reality , 2004, Philosophy of Science.

[61]  Steven French,et al.  Science and Partial Truth: A Unitary Approach to Models and Scientific Reasoning , 2003 .

[62]  Shigeru Nakayama,et al.  Multiple pheromone deposition in ant-based clustering as an ant foraging concept , 2007 .

[63]  Bernardo A. Huberman,et al.  Predicting the Future , 2003, Inf. Syst. Frontiers.

[64]  Wendy S. Parker.  Computer simulation through an error-statistical lens , 2007, Synthese.

[65]  Joshua M. Epstein,et al.  Chapter 12. TOWARD A CONTAINMENT STRATEGY FOR SMALLPOX BIOTERROR: AN INDIVIDUAL-BASED COMPUTATIONAL APPROACH , 2004 .