Dorylus: An Ant Colony Based Tool for Automated Test Case Generation

Automated test generation to cover all branches within a program is a hard task. We present Dorylus, a test suite generation tool that uses ant colony optimisation, guided by coverage. Dorylus constructs a continuous domain over which it conducts independent, multiple objective search that employs a lightweight, dynamic, path-based input dependency analysis. We compare Dorylus with EvoSuite with respect to both coverage and speed using two corpora. The first benchmark contains string based programs, where our results demonstrate that Dorylus improves over EvoSuite on branch coverage and is 50% faster on average. The second benchmark consists of 936 Java programs from SF110 and suggests Dorylus generalises well as it achieves 79% coverage on average whereas the best performing of three EvoSuite algorithms reaches 89%.

[1]  Gordon Fraser,et al.  An empirical evaluation of evolutionary algorithms for unit test suite generation , 2018, Inf. Softw. Technol..

[2]  Marco Dorigo,et al.  Ant colony optimization for continuous domains , 2008, Eur. J. Oper. Res..

[3]  Gordon Fraser,et al.  Whole Test Suite Generation , 2013, IEEE Transactions on Software Engineering.

[4]  Gordon Fraser,et al.  EvoSuite: automatic test suite generation for object-oriented software , 2011, ESEC/FSE '11.

[5]  Gordon Fraser,et al.  An Empirical Evaluation of Evolutionary Algorithms for Test Suite Generation , 2017, SSBSE.

[6]  Gordon Fraser,et al.  Improving search-based test suite generation with dynamic symbolic execution , 2013, 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE).

[7]  Andrea Arcuri,et al.  Test suite generation with the Many Independent Objective (MIO) algorithm , 2018, Inf. Softw. Technol..

[8]  GORDON FRASER,et al.  A Large-Scale Evaluation of Automated Unit Test Generation Using EvoSuite , 2014, ACM Trans. Softw. Eng. Methodol..

[9]  Paolo Tonella,et al.  A large scale empirical comparison of state-of-the-art search-based test case generators , 2018, Inf. Softw. Technol..

[10]  Bogdan Korel,et al.  Automated Software Test Data Generation , 1990, IEEE Trans. Software Eng..

[11]  Gordon Fraser,et al.  EvoSuite at the SBST 2019 Tool Competition , 2019, 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST).

[12]  John A. Clark,et al.  Metrics are fitness functions too , 2004, 10th International Symposium on Software Metrics, 2004. Proceedings..

[13]  Andrea Arcuri,et al.  Many Independent Objective (MIO) Algorithm for Test Suite Generation , 2017, SSBSE.

[14]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[15]  Mohammad Alshraideh,et al.  Search‐based software test data generation for string data using program‐specific search operators , 2006, Softw. Test. Verification Reliab..

[16]  Paolo Tonella,et al.  Reformulating Branch Coverage as a Many-Objective Optimization Problem , 2015, 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST).

[17]  Paolo Tonella,et al.  Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets , 2018, IEEE Transactions on Software Engineering.