Epistemic Opacity, Confirmation Holism and Technical Debt: Computer Simulation in the Light of Empirical Software Engineering

Software-intensive Science, and in particular the method of modelling large and complex systems by means of computer simulation, presents acute dilemmas of epistemic trust. Some authors have contended that simulations are essentially epistemically opaque vis and vis a human agent, others that complex simulation models suffer from an inescapable confirmation holism. We argue that the shortcomings lie in the failure of modellers to adopt sound Software Engineering practices, and that the elevation of computational models into superior epistemic authorities runs counter to principles that are common to both Science and Software Engineering.

[1]  Emilia Mendes,et al.  How Reliable Are Systematic Reviews in Empirical Software Engineering? , 2010, IEEE Transactions on Software Engineering.

[2]  James Fleck,et al.  Contingent knowledge and technology development , 1997 .

[3]  Paul Humphreys,et al.  The philosophical novelty of computer simulation methods , 2009, Synthese.

[4]  Susann Wagenknecht,et al.  Facing the Incompleteness of Epistemic Trust: Managing Dependence in Scientific Practice , 2015 .

[5]  Awm Anthonie Meijers,et al.  The relational ontology of technical artifacts , 2000 .

[6]  Dietmar Pfahl,et al.  Reporting guidelines for controlled experiments in software engineering , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[7]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[8]  Fabio Gadducci,et al.  History and Philosophy of Computing , 2015, IFIP Advances in Information and Communication Technology.

[9]  Leslie Lamport.  The Future of Computing: Logic or Biology , 2003 .

[10]  Barry W. Boehm,et al.  Bayesian Analysis of Empirical Software Engineering Cost Models , 1999, IEEE Trans. Software Eng..

[11]  Miryung Kim,et al.  Validity concerns in software engineering research , 2010, FoSER '10.

[12]  J. Marks,et al.  HOLISM , 1991, Journal of the American Dental Association.

[13]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[14]  Marvin V. Zelkowitz,et al.  Experimental Models for Validating Technology , 1998, Computer.

[15]  Fabio Boschetti,et al.  How Computational Models Predict the Behavior of Complex Systems , 2013 .

[16]  Eric Winsberg,et al.  Science in the Age of Computer Simulation , 2010 .

[17]  Mathias Frisch,et al.  Predictivism and old evidence: a critical look at climate model tuning , 2015 .

[18]  Marvin V. Zelkowitz,et al.  EXPERIMENTAL MODELS FOR VALIDATING COMPUTER TECHNOLOGY , 2001 .

[19]  Frank B. Ryan.  The Electronic Voting System for the United States House of Representatives , 1972, Computer.

[20]  James M. Bieman,et al.  Testing scientific software: A systematic literature review , 2014, Inf. Softw. Technol..

[21]  Ruzanna Chitchyan,et al.  Sustainability Design and Software: The Karlskrona Manifesto , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[22]  I. Hacking,et al.  Representing and Intervening. , 1986 .

[23]  Shari Lawrence Pfleeger,et al.  Software Quality: The Elusive Target , 1996, IEEE Softw..

[24]  Dewayne E. Perry,et al.  A case study in root cause defect analysis , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[25]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[26]  Roger Bradbury,et al.  What is a model, why people don't trust them, and why they should , 2012 .

[27]  Nicola Angius,et al.  The Problem of Justification of Empirical Hypotheses in Software Testing , 2014, Philosophy & Technology.

[28]  Paul Humphreys,et al.  Extending Ourselves: Computational Science, Empiricism, and Scientific Method , 2004 .

[29]  Weider D. Yu.  A software fault prevention approach in coding and root cause analysis , 1998, Bell Labs Technical Journal.

[30]  J. J. Gibson.  The theory of affordances , 1977 .

[31]  James Fleck,et al.  Informal information flow and the nature of expertise in financial services , 2014 .

[32]  David Alan Grier,et al.  Software Engineering: History , 2010, Encyclopedia of Software Engineering.

[33]  W. Cannon.  The Wisdom of the Body , 1932 .

[34]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[35]  Steve M. Easterbrook,et al.  Engineering the Software for Understanding Climate Change , 2009, Computing in Science & Engineering.

[36]  S. Raj Chaudhury,et al.  Postprocessing in Automated Grading Systems, Part 2 , 2009, Computing in Science & Engineering.

[37]  D. Klocke,et al.  Tuning the climate of a global model , 2012 .

[38]  Ward Cunningham,et al.  The WyCash portfolio management system , 1992, OOPSLA '92.

[39]  Joe Dewhurst,et al.  Mechanistic Miscomputation: a Reply to Fresco and Primiero , 2014 .

[40]  Robert L. Nord,et al.  Technical Debt: From Metaphor to Theory and Practice , 2012, IEEE Software.

[41]  Mieke Boon,et al.  In Defense of Engineering Sciences: On the Epistemological Relations Between Science and Technology , 2011 .

[42]  Yuanfang Cai,et al.  A Case Study in Locating the Architectural Roots of Technical Debt , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[43]  Reidar Conradi,et al.  Impact of software engineering research on the practice of software configuration management , 2005, ACM Trans. Softw. Eng. Methodol..

[44]  Raymond Turner,et al.  Programming Languages as Technical Artifacts , 2013, Philosophy & Technology.

[45]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[46]  R. Merton.  The Matthew Effect in Science , 1968, Science.

[47]  D. Kourie,et al.  Towards a Philosophy of Software Development: 40 Years after the Birth of Software Engineering , 2008 .

[48]  Eric Winsberg,et al.  Holism, entrenchment, and the future of climate model pluralism , 2010 .

[49]  Keith Devlin,et al.  Logic and information , 1991 .

[50]  Jack K. Horner,et al.  Software Intensive Science , 2014, Philosophy & Technology.

[51]  Andriy V. Miranskyy,et al.  Characteristics of multiple-component defects and architectural hotspots: a large system case study , 2011, Empirical Software Engineering.

[52]  Steve Easterbrook,et al.  The software architecture of climate models: a graphical comparison of CMIP5 and EMICAR5 configurations , 2015 .

[53]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[54]  Aditya P. Mathur,et al.  Foundations of Software Testing , 2007 .

[55]  Jennifer M. Urban,et al.  Shining Light into Black Boxes , 2012, Science.

[56]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[57]  Jon Pipitone,et al.  Assessing climate model software quality: a defect density analysis of three models , 2012 .

[58]  Walter F. Tichy,et al.  Should Computer Scientists Experiment More? , 1998, Computer.

[59]  Giuseppe Primiero,et al.  On malfunctioning software , 2014, Synthese.