Automatic test Oracle for image processing applications using support vector machines

Software testing has been a challenging job over the decades and possess more challenges for complex inputs such as images. While evaluating correctness of the output images, there may exist a large number of correct or incorrect images with insignificant differences. A test oracle is required to evaluate the correctness of output images which may not be available in most of the cases. Currently, output images are evaluated by domain experts such as medical experts, which involves manual inspection of output images at each step of software development. In this paper, we have proposed a mechanism to automate the test oracle using support vector machine. It requires a few correct and incorrect images for the training and is capable of classification of correct and incorrect output images. For the demonstration purpose, we used different implementations of image dilation and compared the results with statistical oracle and metamorphic testing. The results in our initial experiments are encouraging.

[1]  Johannes Mayer,et al.  Statistical Metamorphic Testing Testing Programs with Random Output by Means of Statistical Hypothesis Tests and Metamorphic Testing , 2007 .

[2]  Mark Harman,et al.  The Oracle Problem in Software Testing: A Survey , 2015, IEEE Transactions on Software Engineering.

[3]  Tong Wu,et al.  Self-Checked Metamorphic Testing of an Image Processing Program , 2010, 2010 Fourth International Conference on Secure Software Integration and Reliability Improvement.

[4]  Leo Grady,et al.  Automating image segmentation verification and validation by learning test oracles , 2011, Inf. Softw. Technol..

[5]  Kwan Yong Sim,et al.  Evaluating the effectiveness of metamorphic testing on edge detection programs , 2013 .

[6]  Hong Shan Neoh,et al.  Adaptive Edge Detection for Real-Time Video Processing using FPGAs , 2005 .

[7]  Johannes Mayer,et al.  On Testing Image Processing Applications with Statistical Methods , 2005, Software Engineering.

[8]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[9]  Johannes Mayer,et al.  On Random Testing of Image Processing Applications , 2006, 2006 Sixth International Conference on Quality Software (QSIC'06).

[10]  Bart Baesens,et al.  Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines , 2006 .

[11]  Mark Harman,et al.  Reducing qualitative human oracle costs associated with automatically generated test data , 2010, STOV '10.

[12]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..

[14]  Ralph Guderlei,et al.  Towards Automatic Testing of Imaging Software by Means of Random and Metamorphic Testing , 2007, Int. J. Softw. Eng. Knowl. Eng..

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  Bart Baesens,et al.  Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines , 2007, Eur. J. Oper. Res..

[17]  Liu Chao,et al.  Test oracles based on metamorphic relations for image processing applications , 2015, 2015 IEEE/ACIS 16th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD).

[18]  William E. Howden,et al.  Theoretical and Empirical Studies of Program Testing , 1978, IEEE Transactions on Software Engineering.

[19]  Sheeva Afshan,et al.  Evolving Readable String Test Inputs Using a Natural Language Model to Reduce Human Oracle Cost , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[20]  Djemel Ziou,et al.  Edge Detection Techniques-An Overview , 1998 .

[21]  C. Chandrasekar,et al.  A Comparison of various Edge Detection Techniques used in Image Processing , 2012 .