Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritization and Testing of Object Oriented Software Products

Testing has been an essential part of software development life cycle. Automatic test case and test data generation has attracted many researchers in the recent past. Test suite generation is the concept given importance which considers multiple objectives in mind and ensures core coverage. The test cases thus generated can have dependencies such as open dependencies and closed dependencies. When there are dependencies, it is obvious that the order of execution of test cases can have impact on the percentage of flaws detected in the software under test. Therefore test case prioritization is another important research area that complements automatic test suite generation in objects oriented systems. Prior researches on test case prioritization focused on dependency structures. However, in this paper, we automate the extraction of dependency structures. We proposed a methodology that takes care of automatic test suite generation and test case prioritization for effective testing of object oriented software. We built a tool to demonstrate the proof of concept. The empirical study with 20 case studies revealed that the proposed tool and underlying methods can have significant impact on the software industry and associated clientele.

[1]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[2]  Joseph Robert Horgan,et al.  A study of effective regression testing in practice , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[3]  Martin Glinz,et al.  Using Dependency Charts to Improve Scenario-Based Testing , 2000 .

[4]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[5]  Gregg Rothermel,et al.  Incorporating varying test costs and fault severities into test case prioritization , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[6]  Gregg Rothermel,et al.  The impact of test suite granularity on the cost-effectiveness of regression testing , 2002, ICSE '02.

[7]  Eda Marchetti,et al.  The Cow_Suite Approach to Planning and Deriving Test Suites in UML Projects , 2002, UML.

[8]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[9]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[10]  Doo-Hwan Bae,et al.  An approach to feature based modelling by dependency alignment for the maintenance of the trustworthy system , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[11]  J. Jenny Li,et al.  Code-coverage guided prioritized test generation , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[12]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[13]  J. Jenny Li,et al.  Prioritize code for testing to improve code coverage of complex software , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[14]  Laurie A. Williams,et al.  System test case prioritization of new and regression test cases , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[15]  Jean-Marc Jézéquel,et al.  Automatic test case optimization: a bacteriologic algorithm , 2005, IEEE Software.

[16]  Paolo Tonella,et al.  Using the Case-Based Ranking Methodology for Test Case Prioritization , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[17]  Myra B. Cohen,et al.  Combinatorial Interaction Regression Testing: A Study of Test Case Generation and Prioritization , 2007, 2007 IEEE International Conference on Software Maintenance.

[18]  Mark Harman,et al.  Search Algorithms for Regression Test Case Prioritization , 2007, IEEE Transactions on Software Engineering.

[19]  Tao Xie,et al.  Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[20]  Jianjun Zhao,et al.  Test Case Prioritization Based on Analysis of Program Structure , 2008, 2008 15th Asia-Pacific Software Engineering Conference.

[21]  Neelam Gupta,et al.  Experiments with test case prioritization using relevant slices , 2008, J. Syst. Softw..

[22]  T. H. Tse,et al.  Adaptive Random Test Case Prioritization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[23]  S. A. Sahaaya Arul Mary,et al.  Factor oriented requirement coverage based system test case prioritization of new and regression test cases , 2009, Inf. Softw. Technol..

[24]  Rajib Mall,et al.  System testing for object‐oriented systems with test case prioritization , 2009, Softw. Test. Verification Reliab..

[25]  Christoph Csallner,et al.  Dsc+Mock: a test case + mock class generator in support of coding against interfaces , 2010, WODA '10.

[26]  Michael D. Ernst,et al.  Combined static and dynamic automated test generation , 2011, ISSTA '11.

[27]  Paolo Tonella,et al.  Symbolic search-based testing , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[28]  Gordon Fraser,et al.  Combining search-based and constraint-based testing , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[29]  Lionel C. Briand,et al.  A practical guide for using statistical tests to assess randomized algorithms in software engineering , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[30]  Lionel C. Briand,et al.  Random Testing: Theoretical Results and Practical Implications , 2012, IEEE Transactions on Software Engineering.

[31]  Andrea Arcuri,et al.  A Theoretical and Empirical Analysis of the Role of Test Sequence Length in Software Testing for Structural Coverage , 2012, IEEE Transactions on Software Engineering.

[32]  Andreas Zeller,et al.  Mutation-Driven Generation of Unit Tests and Oracles , 2010, IEEE Transactions on Software Engineering.

[33]  Myra B. Cohen,et al.  An orchestrated survey of methodologies for automated software test case generation , 2013, J. Syst. Softw..

[34]  Gordon Fraser,et al.  Whole Test Suite Generation , 2013, IEEE Transactions on Software Engineering.

[35]  Ibrahim Habli,et al.  An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software , 2013, IEEE Transactions on Software Engineering.

[36]  Tim Miller,et al.  Using Dependency Structures for Prioritization of Functional Test Suites , 2013, IEEE Transactions on Software Engineering.

[37]  P. Govindarajulu,et al.  A Comprehensive Survey of Recent Developments in Software Testing Methodologies , 2014 .