Testing of Concurrent Programs Using Genetic Algorithms

Noise injection disturbs the scheduling of program threads in order to increase the probability that more of their different legal interleavings occur during the testing process. However, there exist many different types of noise heuristics with many different parameters that are not easy to set such that noise injection is really efficient. In this paper, we propose a new way of using genetic algorithms to search for suitable types of noise heuristics and their parameters. This task is formalized as the test and noise configuration search problem in the paper, followed by a discussion of how to represent instances of this problem for genetic algorithms, which objectives functions to use, as well as parameter tuning of genetic algorithms when solving the problem. The proposed approach is evaluated on a set of benchmarks, showing that it provides significantly better results than the so far preferred random noise injection.

[1]  Yosi Ben-Asher,et al.  Noise Makers Need to Know Where to be Silent – Producing Schedules That Find Bugs , 2006, Second International Symposium on Leveraging Applications of Formal Methods, Verification and Validation (isola 2006).

[2]  Enrique Alba,et al.  Comparing Metaheuristic Algorithms for Error Detection in Java Programs , 2011, SSBSE.

[3]  C. Pasareanu,et al.  Lifecycle Verification of the NASA Ames K 9 Rover Executive , 2004 .

[4]  Enrique Alba,et al.  Finding deadlocks in large concurrent java programs using genetic algorithms , 2008, GECCO '08.

[5]  Elad Yom-Tov,et al.  Instrumenting where it hurts: an automatic concurrent debugging technique , 2007, ISSTA '07.

[6]  Enrique Alba,et al.  Searching for liveness property violations in concurrent systems with ACO , 2008, GECCO '08.

[7]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[8]  Shaz Qadeer,et al.  CHESS: A Systematic Testing Tool for Concurrent Software , 2007 .

[9]  Mark Harman,et al.  Search-based software engineering , 2001, Inf. Softw. Technol..

[10]  Tomás Vojnar,et al.  Coverage Metrics for Saturation-Based and Search-Based Testing of Concurrent Software , 2011, RV.

[11]  Garth A. Gibson,et al.  dBug: Systematic Testing of Unmodified Distributed and Multi-threaded Systems , 2011, SPIN.

[12]  Enrique Alba,et al.  Elementary Landscape Decomposition of the Test Suite Minimization Problem , 2011, SSBSE.

[13]  Zdenek Letko.  Sophisticated Testing of Concurrent Programs , 2010, 2nd International Symposium on Search Based Software Engineering.

[14]  Andreas Podelski,et al.  ACSAR: Software Model Checking with Transfinite Refinement , 2007, SPIN.

[15]  Serdar Tasiran,et al.  Goldilocks: a race and transaction-aware java runtime , 2007, PLDI '07.

[16]  Enrique Alba,et al.  Finding safety errors with ACO , 2007, GECCO '07.

[17]  Eitan Farchi,et al.  Framework for testing multi‐threaded Java programs , 2003, Concurr. Comput. Pract. Exp..

[18]  Eitan Farchi,et al.  Applications of synchronization coverage , 2005, PPoPP.

[19]  Tomás Vojnar,et al.  A platform for search-based testing of concurrent software , 2010, PADTAD '10.

[20]  David R. White.  Software review: the ECJ toolkit , 2011, Genetic Programming and Evolvable Machines.

[21]  Yaniv Eytani.  Concurrent Java Test Generation as a Search Problem , 2006, Electron. Notes Theor. Comput. Sci..

[22]  Tomás Vojnar,et al.  Noise Injection Heuristics for Concurrency Testing , 2011, MEMICS.

[23]  Petr Hliněný,et al.  Mathematical and Engineering Methods in Computer Science , 2012, Lecture Notes in Computer Science.

[24]  Tomás Vojnar,et al.  Healing data races on-the-fly , 2007, PADTAD '07.

[25]  Sarfraz Khurshid,et al.  Exploring very large state spaces using genetic algorithms , 2004, International Journal on Software Tools for Technology Transfer.