What's wrong with social simulations?

/. IntroductionIn this paper I will try to answer the question: Why is the epistemic value of so many social simulations questionable? Under 'social simulations' I understand computer simulations of human interaction as they are studied in the social sciences. The reason why I consider the epistemic value of many social simulations as questionable is that many simulation studies cannot give an answer to the most salient question that any scientific study should be ready to answer: How do we know it's true? or, if specifically directed to simulation studies: How do we know that the simulation correctly simulates the phenomenon that it simulates? Answering this question requires some kind of empirical validation of the simulation. The requirement of empirical validation is in line with the widely accepted notion that science is demarcated from nonscience by its empirical testability or fal- sifiability. Many simulation studies, however, do not offer any suggestion about how they could possibly be validated empirically.A frequent reply by simulation scientists is that no simulation of empirical phenomena was intended, but that the simulation only serves a "theoretical" purpose. Then, however, another equally salient question should be answered: Why should we care about the results? It is my strong impression that many social simulation studies cannot answer either this or the first question. In the social sciences it is hard to find useful examples of the use of computers for purely theoretical purposes. In any case, the social sciences are empirical sciences. Therefore, social simulations should contribute either directly or indirectly to our understanding of social phenomena in the empirical world.There exist many different types of simulations but I will restrict myself to agent-based and game theoretical simulations. I do not make a sharp difference between models and simulations. For the purpose of this paper I identify computer simulations just with programmed models. Most of my criticism of the practice of these simulation types can probably be generalized to other types of simulations or models in the social sciences and maybe also to some instances of the simulation practice in the natural sciences, but it would lead us too far afield to discuss these connections here.In order to bring my point home, I rely on the survey by Heath, Hill, and Ciarallo (2009) on agent-based modeling practice for a general overview, and on two example cases that I examine in detail. I start by discussing the survey which reveals that in an important subfield of social simulations, namely, agent-based simulations, empirical validation is commonly lacking. After that I first discuss Thomas Schelling's wellknown neighborhood segregation model. This is a model that I do not consider as being devoid of epistemic value. For, unlike most social simulations, it can be empirically falsified. The discussion of the particular features that make this model scientifically valuable will help us to understand why the simulation models discussed in the following fail to be so. The simulation models that I discuss in the following section are simulations in the tradition of Robert Axelrod's "Evolution of Cooperation" (Axelrod 1984). Although the modeling tradition initiated by Axelrod has delivered hardly any tenable and empirically applicable results, it still continues to thrive today.Finally, the question remains why scientists continue to produce such an abundance of simulation studies that fail to be empirically applicable. In my opinion the failure to produce useful results has a lot to do with the positivist attitude prevailing in this field of the social sciences. This attitude includes the dogmatic belief in the superiority of the methods of natural sciences like physics in any area of science.2. Simulation Without Validation in Agent-Based ModelsIn this section I give my interpretation of a survey by Heath, Hill, and Ciarallo (2009) on agent-based-simulations. …

[1]  Daniel Taylor,et al.  Evolution of the social contract , 2014 .

[2]  Ken Binmore,et al.  Game Theory and the Social Contract. Vol. II: Just Playing , 1997 .

[3]  Raymond R. Hill,et al.  A Survey of Agent-Based Modeling Practices (January 1998 to July 2008) , 2009, J. Artif. Soc. Soc. Simul..

[4]  H. Roche,et al.  Why Copy Others? Insights from the Social Learning Strategies Tournament , 2010 .

[5]  Flaminio Squazzoni,et al.  Is Social Simulation a Social Science Outstation? A Bibliometric Analysis of the Impact of JASSS , 2013, J. Artif. Soc. Soc. Simul..

[6]  O. Wolkenhauer.  Why model? , 2013, Front. Physiol..

[7]  Julian Reiss,et al.  A Plea for (Good) Simulations: Nudging Economics Toward an Experimental Science , 2011 .

[8]  Robert Sugden,et al.  Credible Worlds, Capacities and Mechanisms , 2009 .

[9]  Uskali Mäki,et al.  MISSing the World. Models as Isolations and Credible Surrogate Systems , 2009 .

[10]  M. Milinski.  TIT FOR TAT in sticklebacks and the evolution of cooperation , 1987, Nature.

[11]  Steven F. Railsback,et al.  Agent-Based and Individual-Based Modeling: A Practical Introduction , 2011 .

[12]  Matthias Meyer,et al.  Typical Pitfalls of Simulation Modeling - Lessons Learned from Armed Forces and Business , 2012, J. Artif. Soc. Soc. Simul..

[13]  Ken Binmore,et al.  Game theory and the social contract , 1984 .

[14]  Robert Hoffmann,et al.  Twenty Years on: The Evolution of Cooperation Revisited , 2000, J. Artif. Soc. Soc. Simul..

[15]  T. Ashworth.  Trench warfare, 1914-1918 , 1980 .

[16]  Jaakko Kuorikoski,et al.  Incredible Worlds, Credible Results , 2009 .

[17]  Robert Sugden,et al.  Credible worlds: the status of theoretical models in economics , 2000 .

[18]  Peter Hammerstein,et al.  Why is reciprocity so rare in social animals? A protestant appeal. , 2003 .

[19]  Katharina Weiss,et al.  Game Theory And Animal Behavior , 2016 .

[20]  Thomas C. Schelling,et al.  Dynamic models of segregation , 1971 .

[21]  Tomas Salamon.  Design of Agent-Based Models , 2011 .

[22]  Robin J. Wilson.  Four Colors Suffice: How the Map Problem Was Solved , 2002 .

[23]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[24]  Robert W. Batterman,et al.  Game Theoretic Explanations and the Evolution of Justice , 1998, Philosophy of Science.

[25]  R. Schüssler.  Kooperation unter Egoisten: Vier Dilemmata , 1997 .

[26]  Geoffrey Parker,et al.  Cooperation under predation risk: a data-based ESS analysis , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[27]  K. Binmore.  Just playing: game theory and the social contract II , 1997 .

[28]  Tomáš Šalamon,et al.  Design of agent-based models : developing computer simulations for a better understanding of social processes , 2011 .

[29]  George W. Baer,et al.  Trench Warfare, 1914-1918: The Live and Let Live System. , 1980 .

[30]  L. Dugatkin.  Cooperation Among Animals: An Evolutionary Perspective , 1997 .

[31]  J. Moses,et al.  Ways of Knowing: Competing Methodologies in Social and Political Research , 2007 .

[32]  Eckhart Arnold,et al.  Simulation Models of the Evolution of Cooperation as Proofs of Logical Possibilities. How Useful Are They , 2013 .

[33]  N. Emrah Aydinonat.  Models, conjectures and exploration: an analysis of Schelling's checkerboard model of residential segregation , 2007 .

[34]  Valerie Gray Hardcastle,et al.  Ways of Knowing , 1996, Consciousness and Cognition.

[35]  Ian Shapiro,et al.  The Flight from Reality in the Human Sciences , 2005 .

[36]  Rainer Hegselmann,et al.  A Replication That Failed - on the Computational Model in 'Michael W. Macy and Yoshimichi Sato: Trust, Cooperation and Market Formation in the U.S. and Japan. Proceedings of the National Academy of Sciences, May 2002' , 2008, J. Artif. Soc. Soc. Simul..

[37]  Ingrid Gould Ellen,et al.  Sharing America’s Neighborhoods: The Prospects for Stable Racial Integration , 2001 .

[38]  G. Nigel Gilbert,et al.  Simulation for the social scientist , 1999 .