The Theory of Composite Faults

Fault masking happens when the effect of one fault serves to mask that of another fault for particular test inputs. The coupling effect is relied upon by testing practitioners to ensure that fault masking is rare. It states that complex faults are coupled to simple faults in such a way that a test data set that detects all simple faults in a program will detect a high percentage of the complex faults. While this effect has been empirically evaluated, our theoretical understanding of the coupling effect is as yet incomplete. Wah proposed a theory of the coupling effect on finite bijective (or near bijective) functions with the same domain and co-domain and assuming a uniform distribution for candidate functions. This model, however, was criticized as being too simple to model real systems, as it did not account for differing domain and co-domain in real programs, or for the syntactic neighborhood. We propose a new theory of fault coupling for general functions (with certain constraints). We show that there are two kinds of fault interactions, of which only the weak interaction can be modeled by the theory of the coupling effect. The strong interaction can produce faults that are semantically different from the original faults. These faults should hence be considered as independent atomic faults. Our analysis shows that the theory holds even when the effect of the syntactic neighborhood of the program is considered. We analyze numerous real-world programs with real faults to validate our hypothesis.

[1]  Jeffrey M. Voas,et al.  Semantic metrics for software testability , 1993, J. Syst. Softw..

[2]  Lionel C. Briand,et al.  Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria , 2006, IEEE Transactions on Software Engineering.

[3]  Kalpesh Kapoor,et al.  Formal Analysis of Coupling Hypothesis for Logical Faults , 2006, Innovations in Systems and Software Engineering.

[4]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[5]  K. S. How Tai Wah,et al.  A theoretical study of fault coupling , 2000, Softw. Test. Verification Reliab..

[6]  Larry J Morell,et al.  A Theory of Fault-Based Testing , 1990, IEEE Trans. Software Eng..

[7]  Martin R. Woodward,et al.  Investigating the Partial Relationships Between Testability and the Dynamic Range-to-Domain Ratio , 2003, Australas. J. Inf. Syst..

[8]  Sven Apel,et al.  Views on Internal and External Validity in Empirical Software Engineering , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[9]  David Clark,et al.  Squeeziness: An information theoretic measure for avoiding fault masking , 2012, Inf. Process. Lett..

[10]  Michael D. Ernst,et al.  Are mutants a valid substitute for real faults in software testing? , 2014, SIGSOFT FSE.

[11]  R WoodwardMartin,et al.  Testability, fault size and the domain-to-range ratio , 2000 .

[12]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[13]  A. Jefferson Offutt,et al.  Investigations of the software testing coupling effect , 1992, TSEM.

[14]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[15]  Alex Groce,et al.  Mutations: How Close are they to Real Faults? , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[16]  Martin R. Woodward,et al.  Testability, fault size and the domain-to-range ratio: An eternal triangle , 2000, ISSTA '00.

[17]  Francisco J. O. Dias Fault Masking in Combinational Logic Circuits , 1975, IEEE Transactions on Computers.

[18]  Mark Harman,et al.  An analysis of the relationship between conditional entropy and failed error propagation in software testing , 2014, ICSE.

[19]  Gregg Rothermel,et al.  On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques , 2006, IEEE Transactions on Software Engineering.

[20]  K. S. How Tai Wah A Theoretical Study of Fault Coupling , 2000, Softw. Test. Verification Reliab..

[21]  K. S. How Tai Wah,et al.  Fault coupling in finite bijective functions , 1995, Softw. Test. Verification Reliab..

[22]  K. S. How Tai Wah Theoretical insights into the coupling effect , 2001 .

[23]  Mark Harman,et al.  A Manifesto for Higher Order Mutation Testing , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[24]  A. Jefferson Offutt,et al.  An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-Uses and Prime Path Coverage , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[25]  A. Offutt The coupling effect: fact or fiction , 1989 .

[26]  K. S. How Tai Wah,et al.  An analysis of the coupling effect I: single test data , 2003, Sci. Comput. Program..

[27]  Mark Harman,et al.  Higher Order Mutation Testing , 2009, Inf. Softw. Technol..