Finding failures from passed test cases: improving the pattern classification approach to the testing of mesh simplification programs

Mesh simplification programs create three-dimensional polygonal models similar to an original polygonal model, and yet use fewer polygons. They produce different graphics even though they are based on the same original polygonal model. This results in a test oracle problem. To address the problem, our previous work has developed a technique that uses a reference model of the program under test to train a classifier. Using such an approach may mistakenly mark a failure-causing test case as passed. It lowers the testing effectiveness of revealing failures. This paper suggests piping the test cases marked as passed by a statistical pattern classification module to an analytical metamorphic testing (MT) module. We evaluate our approach empirically using three subject programs with over 2700 program mutants. The result shows that, using a resembling reference model to train a classifier, the integrated approach can significantly improve the failure detection effectiveness of the pattern classification approach. We also explain how MT in our design trades specificity for sensitivity. Copyright © 2009 John Wiley & Sons, Ltd. The paper proposes an integrated testing methodology that builds atop classification technique (PAT 1) and metamorphic testing (MT) to test mesh simplification programs. Test cases labeled as passed by a classifier trained with resembling reference models are checked via MT to look for missed failures. The methodology trades specificity for sensitivity, and empirically outperforms similar methodologies that either classifiers are trained with dissimilar reference models or there is no metamorphic testing component. Copyright © 2009 John Wiley & Sons, Ltd.