Computational intelligence and safe reduction of test suite

Systems are frequently regression tested during the maintenance phase due to corrective, preventive, adaptive or perfective actions. Regression testing is used to prevent the undesirable effects of these changes on the previously tested version. Due to these changes, new test cases become part of the test suite making it huge and inefficient for `retest all' strategy. The ultimate solution of this problem is optimization or reduction of the test suite. Computational Intelligence (CI) based approaches like evolutionary computation, fuzzy logic, neural networks and swarm optimization have been used for test suite reduction. Optimization approaches reduce the test suite by compromising its safety. Ideally optimization of test suite must guarantee safe reduction. In this work, we have optimized the test suite using some CI based approaches and then analysed the test suite for `safe reduction'. Safe reduction can be gauged using control flow graphs. Test cases of optimal solutions were traversed on these graphs. We found that these solutions partially cover control flow graph. This showed that optimal solutions returned by CI based approaches except fuzzy logic are not safe and will be inadequate for regression testing.

[1]  Nasser Jazdi,et al.  Prioritization of Test Cases Using Software Agents and Fuzzy Logic , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[2]  Dr. K. Vivekanandan,et al.  Improving Regression Testing through Modified Ant Colony Algorithm on a Dependency Injected Test Pattern , 2012 .

[3]  Taghi M. Khoshgoftaar,et al.  Application of fuzzy expert system in test case selection for system regression test , 2005, IRI -2005 IEEE International Conference on Information Reuse and Integration, Conf, 2005..

[4]  David W. Coit,et al.  Multi-objective optimization using genetic algorithms: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[5]  Rodrigo Fernandes de Mello,et al.  A Technique to Reduce the Test Case Suites for Regression Testing Based on a Self-Organizing Neural Network Architecture , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[6]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[7]  Arvinder Kaur,et al.  A Bee Colony Optimization Algorithm for Fault Coverage Based Regression Test Suite Prioritization , 2011 .

[8]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[9]  Arvinder Kaur,et al.  A GENETIC ALGORITHM FOR REGRESSION TEST CASE PRIORITIZATION USING CODE COVERAGE , 2011 .

[10]  Manoj Kumar,et al.  Optimization of Test Cases using Soft Computing Techniques : A Critical Review , 2012 .

[11]  Gaurav Kumar,et al.  Software testing optimization through test suite reduction using fuzzy clustering , 2013, CSI Transactions on ICT.

[12]  S. Nachiyappan,et al.  An evolutionary algorithm for regression test suite reduction , 2010, 2010 International Conference on Communication and Computational Intelligence (INCOCCI).

[13]  Manoj Kumar,et al.  Multi-Faceted Measurement Framework for Test Case Classification and Fitness Evaluation using Fuzzy Logic Based Approach , 2012 .

[14]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[15]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[16]  A. Nadeem,et al.  Test suite optimization using fuzzy logic , 2012, 2012 International Conference on Emerging Technologies.

[17]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[18]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[19]  Arun Sharma,et al.  Towards Multi-Faceted Test Cases Optimization , 2011, J. Softw. Eng. Appl..

[20]  Mark Harman,et al.  Regression Testing Minimisation, Selection and Prioritisation - A Survey , 2009 .

[21]  Rajesh Krishnamoorthi,et al.  Regression Test Suite Prioritization using Genetic Algorithms , 2009 .

[22]  Ricardo B. C. Prudêncio,et al.  A Multi-objective Particle Swarm Optimization for Test Case Selection Based on Functional Requirements Coverage and Execution Effort , 2011, 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence.

[23]  Roger S. Pressman,et al.  Software Engineering: A Practitionerʼs Approach, 7/e , 2009 .

[24]  Gregg Rothermel,et al.  A comparative study of coarse- and fine-grained safe regression test-selection techniques , 2001, TSEM.

[25]  Aftab Ali Haider,et al.  Multiple objective test suite optimization: A fuzzy logic based approach , 2014, J. Intell. Fuzzy Syst..

[26]  M. Clerc,et al.  Particle Swarm Optimization , 2006 .

[27]  Aftab Ali Haider,et al.  On the Fly Test Suite Optimization with FuzzyOptimizer , 2013, 2013 11th International Conference on Frontiers of Information Technology.

[28]  Swarnendu Biswas,et al.  Regression Test Selection Techniques: A Survey , 2011, Informatica.

[29]  Dusan Teodorovic,et al.  Bee Colony Optimization (BCO) , 2009, Innovations in Swarm Intelligence.

[30]  Shailja Gupta,et al.  Regression Testing Using Fuzzy Logic , 2013 .

[31]  John Fulcher,et al.  Computational Intelligence: An Introduction , 2008, Computational Intelligence: A Compendium.

[32]  Mark Harman,et al.  Pareto efficient multi-objective test case selection , 2007, ISSTA '07.

[33]  Bharti Suri,et al.  Implementing Ant Colony Optimization for Test Case Selection and Prioritization , 2011 .

[34]  Kaur Arvinder,et al.  Particle Swarm Optimization with Cross-Over Operator for Prioritization in Regression Testing , 2011 .

[35]  Dr. Arvinder Kaur,et al.  Hybrid Particle Swarm Optimization for Regression Testing , 2011 .

[36]  Thomas Stützle,et al.  Ant Colony Optimization Theory , 2004 .

[37]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[38]  Kalyanmoy Deb,et al.  MULTI-OBJECTIVE FUNCTION OPTIMIZATION USING NON-DOMINATED SORTING GENETIC ALGORITHMS , 1994 .

[39]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.