Agile Metamorphic Model-Based Testing

Testing becomes difficult when we cannot easily determine whether the system delivers the correct result or not. To address this issue, we have developed a framework for automated testing of NASA's DAT system using metamorphic testing principles combined with model based testing. Based on the results from using the framework to test DAT we have determined that this is a cost beneficial solution allowing for comprehensive testing that detects defects without having to develop complex testing infrastructure to determine the oracle. In our approach, DAT queries are automatically generated from a set of metamorphic testing models where each model encodes one or more of the identified equivalences. In addition, we use a scheme for generating time stamps that we use to automatically create time pairs that add another equivalence dimension.

[1]  Christoph Schulze,et al.  An initial evaluation of model-based testing , 2013, 2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW).

[2]  Alex Groce,et al.  Randomized Differential Testing as a Prelude to Formal Verification , 2007, 29th International Conference on Software Engineering (ICSE'07).

[3]  Christoph Schulze,et al.  Model-based testing of NASA's OSAL API — An experience report , 2013, 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE).

[4]  Huai Liu,et al.  How Effectively Does Metamorphic Testing Alleviate the Oracle Problem? , 2014, IEEE Transactions on Software Engineering.

[5]  Ralph Guderlei,et al.  Towards Automatic Testing of Imaging Software by Means of Random and Metamorphic Testing , 2007, Int. J. Softw. Eng. Knowl. Eng..

[6]  Bruno Legeard,et al.  Model-based testing , 2015, Commun. ACM.

[7]  Rance Cleaveland,et al.  Assessing model-based testing: an empirical study conducted in industry , 2014, ICSE Companion.

[8]  Larry Apfelbaum,et al.  Model Based Testing , 1997 .

[9]  Gary Marchionini,et al.  A study on video browsing strategies , 1997 .

[10]  Mikael Lindvall,et al.  Metamorphic Model-Based Testing Applied on NASA DAT -- An Experience Report , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[11]  Mikael Lindvall,et al.  An introduction to agile methods , 2004, Adv. Comput..

[12]  Aldo Dagnino,et al.  Agile Software Development in Large Organizations , 2004, Computer.

[13]  Sergio Segura,et al.  Automated metamorphic testing on the analyses of feature models , 2011, Inf. Softw. Technol..