Possibility of cost reduction by mutant clustering according to the clustering scope

Mutation testing offers developers a good way to improve the quality of a test set. However, the high cost of executing a large number of mutants remains an issue. This paper examines the possibility of reducing the cost of statement‐level mutant clustering by comparing the number of mutant executions with those of expression‐level and block‐level mutant clustering. The goal is to investigate to what extent the clustering scope should be extended. The experimental results using nine real‐world programs show that statement‐level clustering can reduce the mutant executions that are required by expression‐level clustering by 10.51% on average. Block‐level clustering exhibits an unexpected result; the number of mutant executions with block‐level clustering is only 1.06% times less than that with statement‐level clustering. That is, statement‐level clustering is more cost‐effective than block‐level clustering when considering their clustering overheads. A compound expression plays a major role in providing a cost‐reduction effect in statement‐level clustering. With a compound expression, the number of candidate mutants to be clustered in a statement scope increases, and state change can be comprehensively examined, thereby increasing the possibility of cost reduction. © 2018 John Wiley & Sons, Ltd.