Experimental evaluation of a novel equivalence class partition testing strategy

In this paper, a complete model-based equivalence class testing strategy recently developed by the authors is experimentally evaluated. This black-box strategy applies to deterministic systems with infinite input domains and finite internal state and output domains. It is complete with respect to a given fault model. This means that conforming behaviours will never be rejected, and all non-conforming behaviours inside a given fault domain will be uncovered. We investigate the question how this strategy performs for systems under test whose behaviours lie outside the fault domain. Furthermore, a strategy extension is presented, that is based on randomised data selection from input equivalence classes. While this extension is still complete with respect to the given fault domain, it also promises a higher test strength when applied against members outside this domain. This is confirmed by an experimental evaluation that compares mutation coverage achieved by the original and the extended strategy with the coverage obtained by random testing. For mutation generation, not only typical software errors, but also critical HW/SW integration errors are considered. The latter can be caused by mismatches between hardware and software design, even in the presence of totally correct software.