Multi-Objective Test Case Selection: A study of the influence of the Catfish effect on PSO based strategies

During the software testing process many test suites can be generated in order to evaluate and assure the quality of the products. In some cases, the execution of all suites can not fit the available resources (time, people, etc). Hence, automatic Test Case (TC) selection could be used to reduce the suites based on some selection criterion. This process can be treated as an optimization problem, aiming to find a subset of TCs which optimizes one or more objective functions (i.e., selection criteria). In this light, we developed mechanisms for TC selection in context of structural and functional testing. The proposed algorithms consider two objectives simultaneously: maximize branch coverage (or functional requirements coverage) while minimizing execution cost (time). These mechanisms were implemented by deploying multi-objective techniques based on Particle Swarm Optimization (PSO). Additionally, we added the socalled catfish effect into the multi-objective selection algorithms in order to improve their results. The performed experiments revealed the feasibility of the proposed strategies.

[1]  Dr. Arvinder Kaur,et al.  Hybrid Particle Swarm Optimization for Regression Testing , 2011 .

[2]  Li-Yeh Chuang,et al.  Improved binary particle swarm optimization using catfish effect for feature selection , 2011, Expert Syst. Appl..

[3]  Chin-Yu Huang,et al.  Analysis of test suite reduction with enhanced tie-breaking techniques , 2009, Inf. Softw. Technol..

[4]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[5]  Ricardo B. C. Prudêncio,et al.  A Multi-objective Particle Swarm Optimization for Test Case Selection Based on Functional Requirements Coverage and Execution Effort , 2011, 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence.

[6]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[7]  Khaled El-Fakih,et al.  Simulated Annealing and Genetic Algorithms for Optimal Regression Testing , 1999, J. Softw. Maintenance Res. Pract..

[8]  Paulo Borba,et al.  Using Process Simulation to Assess the Test Design Effort Reduction of a Model-Based Testing Approach , 2008, ICSP.

[9]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[10]  Rajiv Gupta,et al.  A methodology for controlling the size of a test suite , 1990, Proceedings. Conference on Software Maintenance 1990.

[11]  Joachim Wegener,et al.  Applying particle swarm optimization to software testing , 2007, GECCO '07.

[12]  Rudolf Ramler,et al.  Economic perspectives in test automation: balancing automated and manual testing with opportunity cost , 2006, AST '06.

[13]  Mark Harman.  Making the Case for MORTO: Multi Objective Regression Test Optimization , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[14]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[15]  Mark Harman,et al.  Using hybrid algorithm for Pareto efficient multi-objective test suite minimisation , 2010, J. Syst. Softw..

[16]  R. A. Santana,et al.  A Multiple Objective Particle Swarm Optimization Approach Using Crowding Distance and Roulette Wheel , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[17]  Nicholas Nethercote,et al.  Valgrind: A Program Supervision Framework , 2003, RV@CAV.

[18]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[19]  Mauro Pezzè,et al.  Software testing and analysis - process, principles and techniques , 2007 .

[20]  Mark Harman,et al.  Pareto efficient multi-objective test case selection , 2007, ISSTA '07.

[21]  Cheng-qing Ye,et al.  Test-Suite Reduction Using Genetic Algorithm , 2005, APPT.

[22]  Augusto Sampaio,et al.  Testing Techniques in Software Engineering, Second Pernambuco Summer School on Software Engineering, PSSE 2007, Recife, Brazil, December 3-7, 2007, Revised Lectures , 2010, PSSE.

[23]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[24]  R. Prudêncio,et al.  Search based constrained test case selection using execution effort , 2013, Expert Syst. Appl..

[25]  Brad Clement,et al.  Automated Test Case Selection for Flight Systems using Genetic Algorithms , 2010 .

[26]  Mark Harman,et al.  Highly Scalable Multi Objective Test Suite Minimisation Using Graphics Cards , 2011, SSBSE.

[27]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[28]  Ricardo B. C. Prudêncio,et al.  A Constrained Particle Swarm Optimization Approach for Test Case Selection , 2010, SEKE.

[29]  Mark Harman,et al.  Faster Fault Finding at Google Using Multi Objective Regression Test Optimisation , 2011 .

[30]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.