Test Coverage Optimization for Large Code Problems

Because running all previous tests for the regression testing of a system is time-consuming, the size of a test suite of the system must be reduced intelligently with adequate test coverage and without compromising its fault detection capability. Five algorithms were designed for reducing the size of test suites where two metrics, test's function reach ability and function's test intensity, were defined. Approaches to the algorithm CW-NumMin, CW-CostMin, or CW-CostCov-B are the safe-mode of test case selection with full-modified function coverage, while the CW-CovMax algorithm is of non-safe mode, which was performed under time restriction. In this study, the most efficient algorithm could reduce the cost (time) of a test suite down to 1.10%, on the average, over the MPLS area of Cisco IOS.

[1]  Elaine J. Weyuker,et al.  The distribution of faults in a large industrial software system , 2002, ISSTA '02.

[2]  Gregg Rothermel,et al.  Modeling the cost-benefits tradeoffs for regression testing techniques , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[3]  Cheng-qing Ye,et al.  A genetic algorithm for test-suite reduction , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[4]  Tsong Yueh Chen,et al.  A simulation study on some heuristics for test suite reduction , 1998, Inf. Softw. Technol..

[5]  Cagatay Catal,et al.  Software fault prediction: A literature review and current trends , 2011, Expert Syst. Appl..

[6]  Gregg Rothermel,et al.  A framework for evaluating regression test selection techniques , 1994, Proceedings of 16th International Conference on Software Engineering.

[7]  Gregg Rothermel,et al.  Selecting tests and identifying test coverage requirements for modified software , 1994, ISSTA '94.

[8]  Hong Mei,et al.  An experimental comparison of four test suite reduction techniques , 2006, ICSE.

[9]  Hareton K. N. Leung,et al.  Insights into regression testing (software testing) , 1989, Proceedings. Conference on Software Maintenance - 1989.

[10]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[11]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[12]  Neelam Gupta,et al.  Test suite reduction with selective redundancy , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[13]  Hareton Leung,et al.  A firewall concept for both control-flow and data-flow in regression integration testing , 1992, Proceedings Conference on Software Maintenance 1992.

[14]  Gregg Rothermel,et al.  A safe, efficient regression test selection technique , 1997, TSEM.

[15]  Mary Lou Soffa,et al.  A methodology for controlling the size of a test suite , 1993, TSEM.

[16]  Mary Lou Soffa,et al.  Interprocedual data flow testing , 1989 .

[17]  Nashat Mansour,et al.  Simulated Annealing and Genetic Algorithms for Optimal Regression Testing , 1999, J. Softw. Maintenance Res. Pract..

[18]  Tsong Yueh Chen,et al.  A new heuristic for test suite reduction , 1998, Inf. Softw. Technol..

[19]  Mary Jean Harrold,et al.  Testing evolving software , 1999, J. Syst. Softw..

[20]  Gregg Rothermel,et al.  An empirical study of the effects of minimization on the fault detection capabilities of test suites , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[21]  David S. Rosenblum,et al.  TESTTUBE: a system for selective regression testing , 1994, Proceedings of 16th International Conference on Software Engineering.

[22]  Elaine J. Weyuker,et al.  Predicting the location and number of faults in large software systems , 2005, IEEE Transactions on Software Engineering.

[23]  Yuan-Cheng Lai,et al.  Test Coverage Optimization for Large Code Problems , 2012, 2012 26th International Conference on Advanced Information Networking and Applications Workshops.

[24]  Gregg Rothermel,et al.  Analyzing Regression Test Selection Techniques , 1996, IEEE Trans. Software Eng..

[25]  Hareton K. N. Leung,et al.  Insights into testing and regression testing global variables , 1990, J. Softw. Maintenance Res. Pract..

[26]  Rajiv Gupta,et al.  A methodology for controlling the size of a test suite , 1990, Proceedings. Conference on Software Maintenance 1990.

[27]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[28]  L. Williams,et al.  Regression Test Selection for Black-box Dynamic Link Library Components , 2007, Second International Workshop on Incorporating COTS Software into Software Systems: Tools and Techniques (IWICSS '07).

[29]  Emanuel Melachrinoudis,et al.  Bi-criteria models for all-uses test suite reduction , 2004, Proceedings. 26th International Conference on Software Engineering.

[30]  Gregg Rothermel,et al.  Empirical studies of test‐suite reduction , 2002, Softw. Test. Verification Reliab..

[31]  Joseph Robert Horgan,et al.  Test set size minimization and fault detection effectiveness: a case study in a space application , 1997, Proceedings Twenty-First Annual International Computer Software and Applications Conference (COMPSAC'97).

[32]  Gregg Rothermel,et al.  A safe, efficient algorithm for regression test selection , 1993, 1993 Conference on Software Maintenance.

[33]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[34]  Neelam Gupta,et al.  Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction , 2007, IEEE Transactions on Software Engineering.

[35]  W. Eric Wong,et al.  Effect of test set minimization on fault detection effectiveness , 1998 .