Software robustness: a survey, a theory, and prospects

If a software execution is disrupted, witnessing the execution at a later point may see evidence of the disruption or not. If not, we say the disruption failed to propagate. One name for this phenomenon is software robustness but it appears in different contexts in software engineering with different names. Contexts include testing, security, reliability, and automated code improvement or repair. Names include coincidental correctness, correctness attraction, transient error reliability. As witnessed, it is a dynamic phenomenon but any explanation with predictive power must necessarily take a static view. As a dynamic/static phenomenon it is convenient to take a statistical view of it which we do by way of information theory. We theorise that for failed disruption propagation to occur, a necessary condition is that the code region where the disruption occurs is composed with or succeeded by a subsequent code region that suffers entropy loss over all executions. The higher is the entropy loss, the higher the likelihood that disruption in the first region fails to propagate to the downstream observation point. We survey different research silos that address this phenomenon and explain how the theory might be exploited in software engineering.

[1]  David Clark,et al.  Squeeziness: An information theoretic measure for avoiding fault masking , 2012, Inf. Process. Lett..

[2]  Mark Harman,et al.  Genetic Improvement of Software: A Comprehensive Survey , 2018, IEEE Transactions on Evolutionary Computation.

[3]  Martin Monperrus,et al.  Principles of Antifragile Software , 2014, Programming.

[4]  Westley Weimer,et al.  Software mutational robustness , 2012, Genetic Programming and Evolvable Machines.

[5]  Gordon C. Osbourn,et al.  Towards an Approach to Overcome Software Brittleness , 1999 .

[6]  Mark Harman,et al.  An analysis of the relationship between conditional entropy and failed error propagation in software testing , 2014, ICSE.

[7]  Daniel Jackson,et al.  Chopping: A Generalization of Slicing , 1994 .

[8]  Yuriy Brun,et al.  The plastic surgery hypothesis , 2014, SIGSOFT FSE.

[9]  A. Jefferson Offutt,et al.  How strong is weak mutation? , 1991, TAV4.

[10]  P. Parrend,et al.  Software is Not Fragile , 2016 .

[11]  Philippe Preux,et al.  Correctness attraction: a study of stability of software behavior under runtime perturbation , 2016, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[12]  Tom Adams Bookshelf , 1996, IEEE Softw..

[13]  Benoit Baudry,et al.  A journey among Java neutral program variants , 2019, Genetic Programming and Evolvable Machines.

[14]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..

[15]  Ali Shahrokni,et al.  A systematic review of software robustness , 2013, Inf. Softw. Technol..

[16]  Carl E. Landwehr,et al.  Basic concepts and taxonomy of dependable and secure computing , 2004, IEEE Transactions on Dependable and Secure Computing.

[17]  An Empirical Study on Failed Error Propagation in Java Programs with Real Faults , 2020, ArXiv.

[18]  Robert M. Hierons Avoiding coincidental correctness in boundary value analysis , 2006, TSEM.

[19]  Michael Eonsuk Shin Self-healing components in robust software architecture for concurrent and distributed systems , 2005, Sci. Comput. Program..

[20]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[21]  Wes Masri,et al.  Coincidental correctness in the Defects4J benchmark , 2018, Softw. Test. Verification Reliab..

[22]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .