Automated harvesting of test oracles for reliability testing

Reliability testing is of even greater importance in component-based development than it is in traditional development methods, since it is the only practical way of measuring how well assembled components match each others' expectations. However, support for reliability testing has improved little over the last few years and it remains one of the most costly elements of software engineering. In this paper we present an approach that tackles the most fundamental obstacle to more efficient reliability testing - namely, the problem of automating the generation of "correct" reference results against which to evaluate component outputs. The approach employs the same basic technologies that can be used to discover components for reuse, but instead of building them into the final system it uses them, in combination with others, as an automated pseudo-oracle for measuring the reliability of self-built components. This significantly enhances the utility of component discovery technology, since it allows value to be derived from components which do not pass the threshold needed for full reuse but are sufficiently reliable to form a functioning component of a test oracle.