Multi-threaded Testing with AOP Is Easy, and It Finds Bugs!

We investigate the suitability of AOP (Aspect Oriented Programming) for testing tools by trying to implement the ConTest testing tool using AspectJ, a tool that implements AOP for the Java programming language. We examine whether the entire set of features can be implemented this way, in the context of the larger problem where moving to a higher level of abstraction means that some details cannot be implemented. Our conclusion from this exercise is that AOP is very suitable for the implementation of a number of classes of test tools. These include multi-threaded noise makers such as ConTest, in addition to coverage analyzers, data-race detectors, network traffic simulators, runtime bug pattern detectors, and others. The main advantage is that the instrumentation part of the tool creating method, which usually contains little scientific contribution but consumes most of the work, becomes much easier to perform and requires less expertise. In our specific exercise, a task that took more than half a year and required specialized knowledge, was reduced to two weeks work by a relative novice.