Performance Testing Support in a Continuous Integration Infrastructure

The application of continuous integration allows an agile software development process by automating the build process, so it is nowadays seen as a good practice. However, if the build process is au- tomatized, also the testing has to be automatized, to ensure quality and to detect faults while integrating. One possible quality characteristic of a software that could be checked is its performance. In this paper, we propose a performance testing framework for Java that executes performance tests by using microbenchmarks. It uses the Java benchmarking toolkit JMH and can test for machine-dependent performance assertions. This framework can be included into the continuous integration server Jenkins, so performance tests will be executed automatically during the build process. We conducted an feasibility evaluation of this approach by applying it to the continuous integration infrastructure of the Pipe-and-Filter framework TeeTime.