Dirt Spot Sweeping Random Strategy

The paper presents a new and improved automated random testing technique named as Dirt Spot Sweeping Random (DSSR) strategy based on the rationale that, "when failures lies in the contiguous locations across the input domain, the effectiveness of random testing can be further improved through diversity of test cases". The DSSR strategy selects neighboring values for the subsequent tests on identification of failure. Resultantly, selected values sweep around the failure leading to the discovery of new failures in the vicinity. To evaluate the effectiveness of DSSR strategy a total of 60 classes (35,785 lines of code), each class with 30 x 105 calls, were tested by Random (R), Random+ (R+) and DSSR strategies. T-Test analysis showed significantly better performance of DSSR compared to R strategy in 17 classes and R+ strategy in 9 classes. In the remaining classes all the three strategies performed equally well. Numerically, the DSSR strategy found 43 and 12 more unique failures than R and R+ strategies respectively. This study comprehends that DSSR strategy will have a profound positive impact on the failure- finding ability of R and R+ testing.

[1]  Bertrand Meyer,et al.  Finding Faults: Manual Testing vs. Random+ Testing vs. User Reports , 2008, 2008 19th International Symposium on Software Reliability Engineering (ISSRE).

[2]  Manuel Oriol,et al.  Testing .NET Code with YETI , 2010, 2010 15th IEEE International Conference on Engineering of Complex Computer Systems.

[3]  Bertrand Meyer,et al.  Reconciling Manual and Automated Testing: The AutoTest Experience , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[4]  I. K. Mak,et al.  Adaptive Random Testing , 2004, ASIAN.

[5]  Tsong Yueh Chen,et al.  Proportional sampling strategy: guidelines for software testing practitioners , 1996, Inf. Softw. Technol..

[6]  Bertrand Meyer,et al.  On the number and nature of faults found by random testing , 2011, Softw. Test. Verification Reliab..

[7]  Simeon C. Ntafos,et al.  A report on random testing , 1981, ICSE '81.

[8]  Bertrand Meyer,et al.  Experimental assessment of random testing for object-oriented software , 2007, ISSTA '07.

[9]  Bertrand Meyer,et al.  ARTOO , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[10]  Tsong Yueh Chen,et al.  On the Expected Number of Failures Detected by Subdomain Testing and Random Testing , 1996, IEEE Trans. Software Eng..

[11]  Bertrand Meyer,et al.  On the Effectiveness of Test Extraction without Overhead , 2009, 2009 International Conference on Software Testing Verification and Validation.

[12]  Jing Li,et al.  The Qualitas Corpus: A Curated Collection of Java Code for Empirical Studies , 2010, 2010 Asia Pacific Software Engineering Conference.

[13]  Tsong Yueh Chen,et al.  Adaptive Random Testing: The ART of test case diversity , 2010, J. Syst. Softw..

[14]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2011, SIGP.

[15]  Michael D. Ernst,et al.  Randoop: feedback-directed random testing for Java , 2007, OOPSLA '07.

[16]  Tsong Yueh Chen,et al.  Is adaptive random testing really better than random testing , 2006, RT '06.

[17]  Walter J. Gutjahr,et al.  Partition Testing vs. Random Testing: The Influence of Uncertainty , 1999, IEEE Trans. Software Eng..

[18]  Tsong Yueh Chen,et al.  Quasi-Random Testing , 2005, IEEE Transactions on Reliability.

[19]  Bertrand Meyer,et al.  On the Predictability of Random Tests for Object-Oriented Software , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[20]  Tsong Yueh Chen,et al.  Adaptive random testing through dynamic partitioning , 2004 .

[21]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[22]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[23]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[24]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[25]  Dave Towey,et al.  Restricted Random Testing , 2002, ECSQ.

[26]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[27]  Tsong Yueh Chen,et al.  Mirror adaptive random testing , 2004, Inf. Softw. Technol..

[28]  Mark Harman,et al.  Test data regeneration: generating new test data from existing test data , 2012, Softw. Test. Verification Reliab..

[29]  Michael D. Ernst,et al.  Eclat: Automatic Generation and Classification of Test Inputs , 2005, ECOOP.

[30]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[31]  R. Taylor,et al.  Partition testing does not inspire confidence , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[32]  Lionel C. Briand,et al.  Random Testing: Theoretical Results and Practical Implications , 2012, IEEE Transactions on Software Engineering.

[33]  Simeon C. Ntafos,et al.  On Comparisons of Random, Partition, and Proportional Partition Testing , 2001, IEEE Trans. Software Eng..

[34]  Manuel Oriol Random Testing: Evaluation of a Law Describing the Number of Faults Found , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[35]  Catherine Oriat,et al.  Jartege: A Tool for Random Generation of Unit Tests for Java Classes , 2004, QoSA/SOQUA.