T3 @SBST2018 Benchmark, and How Much We Can Get from Asemantical Testing

This paper discusses the performance of the automated testing tool for Java called T3 and compares it with few other tools and human written tests in a benchmark set by the Java Unit Testing Tool Contest 2018. Since all the compared tools rely on randomisation when generating their test data, albeit to different degrees and with different heuristics, this paper also tries to give some insight on just how far we can go without having to reconstruct the precise semantic of the programs under test in order to test them.

[1]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[2]  Ignatius S.W.B. Prasetya.  Budget-Aware Random Testing with T3: Benchmarking at the SBST2016 Testing Tool Contest , 2016, 2016 IEEE/ACM 9th International Workshop on Search-Based Software Testing (SBST).

[3]  K. Rustan M. Leino,et al.  Checking Java Programs via Guarded Commands , 1999, ECOOP Workshops.

[4]  Yann-Gaël Guéhéneuc,et al.  Instance Generator and Problem Representation to Improve Object Oriented Code Coverage , 2015, IEEE Transactions on Software Engineering.

[5]  Edsger W. Dijkstra,et al.  Guarded commands, nondeterminacy and formal derivation of programs , 1975, Commun. ACM.

[6]  Gordon Fraser,et al.  EvoSuite: automatic test suite generation for object-oriented software , 2011, ESEC/FSE '11.

[7]  Urko Rueda,et al.  Java Unit Testing Tool Competition - Seventh Round , 2019, 2019 IEEE/ACM 12th International Workshop on Search-Based Software Testing (SBST).

[8]  Tanja E. J. Vos,et al.  Trace-based Reflexive Testing of OO Programs with T2 , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[9]  Arie Middelkoop,et al.  Functional Instrumentation of ActionScript Programs with Asil , 2011, IFL.

[10]  Koushik Sen,et al.  CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools , 2006, CAV.

[11]  Annibale Panichella,et al.  Java Unit Testing Tool Competition - Sixth Round , 2018, 2018 IEEE/ACM 11th International Workshop on Search-Based Software Testing (SBST).

[12]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).