Metamorphic Testing: Testing the Untestable

What if we could know that a program is buggy, even if we could not tell whether or not its observed output is correct? Metamorphic testing provides this ability. This article explains the basics of the technique.

[1]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..

[2]  Mikael Lindvall,et al.  Metamorphic Model-Based Testing Applied on NASA DAT -- An Experience Report , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[3]  Tsong Yueh Chen,et al.  Metamorphic Testing: A New Approach for Generating Next Test Cases , 2020, ArXiv.

[4]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[5]  Christoph Schulze,et al.  Metamorphic Model-Based Testing of Autonomous Systems , 2017, 2017 IEEE/ACM 2nd International Workshop on Metamorphic Testing (MET).

[6]  Suman Jana,et al.  DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[7]  Huai Liu,et al.  Metamorphic Testing , 2018, ACM Comput. Surv..

[8]  Lu Zhang,et al.  Search-based inference of polynomial metamorphic relations , 2014, ASE.

[9]  Automated Software Engineering , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[10]  Tsong Yueh Chen,et al.  METRIC: METamorphic Relation Identification based on the Category-choice framework , 2016, J. Syst. Softw..

[11]  Baowen Xu,et al.  Testing and validating machine learning classifiers by metamorphic testing , 2011, J. Syst. Softw..

[12]  Zhendong Su,et al.  Compiler validation via equivalence modulo inputs , 2014, PLDI.

[13]  Jeffrey M. Voas,et al.  Metamorphic Testing for Cybersecurity , 2016, Computer.

[14]  Sergio Segura,et al.  Metamorphic Testing of RESTful Web APIs , 2018, IEEE Transactions on Software Engineering.

[15]  Alastair F. Donaldson,et al.  Automated testing of graphics shader compilers , 2017, Proc. ACM Program. Lang..

[16]  James M. Bieman,et al.  Predicting metamorphic relations for testing scientific software: a machine learning approach using graph kernels , 2016, Softw. Test. Verification Reliab..

[17]  Sergio Segura,et al.  A Survey on Metamorphic Testing , 2016, IEEE Transactions on Software Engineering.

[18]  V. D.,et al.  Prof , 1973 .

[19]  Huai Liu,et al.  How Effectively Does Metamorphic Testing Alleviate the Oracle Problem? , 2014, IEEE Transactions on Software Engineering.

[20]  Tsong Yueh Chen,et al.  Metamorphic Testing for Software Quality Assessment: A Study of Search Engines , 2016, IEEE Transactions on Software Engineering.