Towards Automatic Testing of Imaging Software by Means of Random and Metamorphic Testing

Testing of imaging software is a challenging task, which is usually done manually. For this purpose, well-known test images are generally used whose expected output can be specified in advance or the actual result is visually inspected by the tester. In the present paper, an approach is described that allows to test imaging software fully automatically. Several random models are proposed for test data generation. Metamorphic relations are presented that can be used to generate follow-up test cases and evaluate the result. The models for test data generation and the oracle solutions are compared using mutation analysis. The presented approach is quite generally applicable in the field of imaging software.

[1]  Arnaud Gotlieb.  Exploiting symmetries to test programs , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[2]  Barton P. Miller,et al.  An empirical study of the robustness of MacOS applications using random testing , 2006, RT '06.

[3]  Tsong Yueh Chen,et al.  Metamorphic testing of programs on partial differential equations: a case study , 2002, Proceedings 26th Annual International Computer Software and Applications.

[4]  David G. Kirkpatrick,et al.  Linear Time Euclidean Distance Algorithms , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Claude Caci,et al.  Testing object-oriented systems , 2000, SOEN.

[6]  Johannes Mayer,et al.  On Random Testing of Image Processing Applications , 2006, 2006 Sixth International Conference on Quality Software (QSIC'06).

[7]  Tsong Yueh Chen,et al.  Fault-based testing in the absence of an oracle , 2001, 25th Annual International Computer Software and Applications Conference. COMPSAC 2001.

[8]  Volker Schmidt,et al.  A unified simulation framework for spatial stochastic models , 2004, Simul. Model. Pract. Theory.

[9]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..

[10]  Felix T.S. Chan,et al.  Application of metamorphic testing in numerical analysis , 1998, ICSE 1998.

[11]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[12]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[13]  Johannes Mayer,et al.  Test Oracles Using Statistical Methods , 2004, SOQUA/TECOS.

[14]  Tsong Yueh Chen,et al.  Case studies on the selection of useful relations in metamorphic testing , 2004 .

[15]  Zongyuan Yang,et al.  Metamorphic Testing and Its Applications , 2004 .

[16]  Johannes Mayer,et al.  Testing randomized software by means of statistical hypothesis tests , 2007, SOQUA '07.

[17]  Johannes Mayer,et al.  On Testing Image Processing Applications with Statistical Methods , 2005, Software Engineering.

[18]  P. Danielsson.  Euclidean distance mapping , 1980 .

[19]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[20]  Donald R. Slutz,et al.  Massive Stochastic Testing of SQL , 1998, VLDB.

[21]  Takahide Yoshikawa,et al.  Random program generator for Java JIT compiler test system , 2003, Third International Conference on Quality Software, 2003. Proceedings..

[22]  Johannes Mayer,et al.  An Empirical Study on the Selection of Good Metamorphic Relations , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[23]  Wolfgang Weil,et al.  Densities for stationary random sets and point processes , 1984, Advances in Applied Probability.

[24]  Ying Liu,et al.  Metamorphic Testing and Testing with Special Values , 2004, SNPD.

[25]  Tsong Yueh Chen,et al.  A metamorphic approach to integration testing of context-sensitive middleware-based applications , 2005, Fifth International Conference on Quality Software (QSIC'05).

[26]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[27]  Christian Ronse,et al.  Generation of Shading-Off in Images by Extrapolation of Lipschitz Functions , 1996, CVGIP Graph. Model. Image Process..