Genetic algorithm based test data generation for MPI parallel programs with blocking communication

Abstract Parallel computing is one of mainstream techniques for high-performance computation in which MPI parallel programs have gained more and more attention. Genetic algorithms (GAs) have been widely employed in automated test data generation, leading to a major family of search-based software testing techniques. However, previous GA-based methods have limitations when testing MPI parallel programs with blocking communication. In this paper, we focus on the path coverage problem for MPI parallel programs with blocking communication, and formulate the problem as an optimization problem with its decision variable being the program input and the execution order of sending nodes. In addition, we develop target amending strategies for candidates when solving the problem using genetic algorithms. The proposed method is evaluated and compared with several state-of-the-art methods through a series of controlled experiments on five typical programs. The experimental results show that the proposed method can effectively and efficiently generate test data for path coverage.

[1]  Andrea Arcuri.  Longer is Better: On the Role of Test Sequence Length in Software Testing , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[2]  Cyrille Artho,et al.  Modular Software Model Checking for Distributed Systems , 2014, IEEE Transactions on Software Engineering.

[3]  Dunwei Gong,et al.  Evolutionary generation of test data for many paths coverage , 2010, 2010 Chinese Control and Decision Conference.

[4]  Manju Khari,et al.  Heuristic search-based approach for automated test data generation: a survey , 2013, Int. J. Bio Inspired Comput..

[5]  Tomás Vojnar,et al.  Testing of Concurrent Programs Using Genetic Algorithms , 2012, SSBSE.

[6]  Gordon Fraser,et al.  Generating Unit Tests for Concurrent Classes , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[7]  Kalim Qureshi.  A Practical Performance Comparison of Parallel Sorting Algorithms on Homogeneous Network of Workstations , 2006, PDPTA.

[8]  Rajeev Thakur,et al.  Implementing Efficient Dynamic Formal Verification Methods for MPI Programs , 2008, PVM/MPI.

[9]  Lionel C. Briand,et al.  A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation , 2010, IEEE Transactions on Software Engineering.

[10]  Mario Jino,et al.  Diversity oriented test data generation using metaheuristic search techniques , 2014, Inf. Sci..

[11]  Gordon Fraser,et al.  On Parameter Tuning in Search Based Software Engineering , 2011, SSBSE.

[12]  Simone do Rocio Senger de Souza,et al.  Mutation operators for concurrent programs in MPI , 2012, 2012 13th Latin American Test Workshop (LATW).

[13]  Silvia Regina Vergilio,et al.  Structural testing criteria for message-passing parallel programs , 2008 .

[14]  William Gropp,et al.  Mpi---the complete reference: volume 1 , 1998 .

[15]  Yuanyuan Zhang,et al.  Search-based software engineering: Trends, techniques and applications , 2012, CSUR.

[16]  Ed Zaluska,et al.  Structural testing for message‐passing concurrent programs: an extended test model , 2014, Concurr. Comput. Pract. Exp..

[17]  Mark Harman,et al.  The species per path approach to SearchBased test data generation , 2006, ISSTA '06.

[18]  Jesper Larsson Träff,et al.  SKaMPI: a comprehensive benchmark for public benchmarking of MPI , 2002, Sci. Program..

[19]  Irman Hermadi,et al.  GA-based multiple paths test data generator , 2008, Comput. Oper. Res..

[20]  Michael M. Resch,et al.  Correctness Checking of MPI One-Sided Communication Using Marmot , 2006, PVM/MPI.

[21]  Zengo Furukawa,et al.  A test-case generation method for concurrent programs including task-types , 1997, Proceedings of Joint 4th International Computer Science Conference and 4th Asia Pacific Software Engineering Conference.

[22]  Jeff Huang,et al.  CLAP: recording local executions to reproduce concurrency failures , 2013, PLDI.

[23]  Gregg Rothermel,et al.  An experimental determination of sufficient mutant operators , 1996, TSEM.

[24]  Wei Dong,et al.  MPISE: Symbolic Execution of MPI Programs , 2014, 2015 IEEE 16th International Symposium on High Assurance Systems Engineering.

[25]  Yaniv Eytani.  Concurrent Java Test Generation as a Search Problem , 2006, Electron. Notes Theor. Comput. Sci..

[26]  Zuohua Ding,et al.  Test Case Generation of Concurrent Programs Based on Event Graph , 2008, 2009 Fifth International Joint Conference on INC, IMS and IDC.

[27]  Bronis R. de Supinski,et al.  Dynamic Software Testing of MPI Applications with Umpire , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[28]  Vahid Garousi,et al.  Empirical analysis of a genetic algorithm-based stress test technique , 2008, GECCO '08.

[29]  Dun-Wei Gong,et al.  Test data generation for path coverage of message-passing parallel programs based on co-evolutionary genetic algorithms , 2014, Automated Software Engineering.

[30]  Enrique Alba,et al.  Finding deadlocks in large concurrent java programs using genetic algorithms , 2008, GECCO '08.

[31]  Xiao Ma,et al.  Effective generation of test sequences for structural testing of concurrent programs , 2005, 10th IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'05).

[32]  Orna Grumberg,et al.  Bounded Model Checking of Concurrent Programs , 2005, CAV.

[33]  Mark Harman,et al.  A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search , 2010, IEEE Transactions on Software Engineering.

[34]  Changhai Nie,et al.  A Discrete Particle Swarm Optimization for Covering Array Generation , 2015, IEEE Transactions on Evolutionary Computation.

[35]  Konstantinos Sagonas,et al.  Detection of Asynchronous Message Passing Errors Using Static Analysis , 2011, PADL.

[36]  Abdul Rauf,et al.  Automated GUI Test Coverage Analysis Using GA , 2010, 2010 Seventh International Conference on Information Technology: New Generations.