Model-based replay testing for event-driven software

Event-driven software systems are hard to thoroughly test because they possess a huge number of possible event sequences to test and also require significant manual effort and/or special hardware support during test case execution. In this paper we propose a novel testing approach, we call model-based replay testing that combines the advantages of model-based testing and an operating system (OS) level replay mechanism to overcome such difficulties of event-driven software testing. Specifically, we use a finite state machine (FSM) model and an n-switch set cover to automatically generate abstract test cases. We then introduce the use of OS-level replay mechanism in transforming abstract test cases into concrete test cases and executing them in an automated manner. To demonstrate the effectiveness of the proposed approach, we also describe a case study on GUI software testing using a prototype implementation of an OS-level replay mechanism in Android software platform.

[1]  Alessandra Gorla,et al.  Automated Test Input Generation for Android: Are We There Yet? (E) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[2]  Nina Yevtushenko,et al.  An Improved Conformance Testing Method , 2005, FORTE.

[3]  Dianxiang Xu,et al.  A Systematic Capture and Replay Strategy for Testing Complex GUI Based Java Applications , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[4]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[5]  Mukul R. Prasad,et al.  Automated testing with targeted event sequence generation , 2013, ISSTA.

[6]  Sam Malek,et al.  EvoDroid: segmented evolutionary testing of Android apps , 2014, SIGSOFT FSE.

[7]  Gerard J. Holzmann,et al.  A practical method for verifying event-driven software , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[8]  István Majzik,et al.  Model-based Automatic Test Generation for Event-Driven Embedded Systems using Model Checkers , 2006, 2006 International Conference on Dependability of Computer Systems.

[9]  Minsoo Ryu,et al.  RT-replayer: a record-replay architecture for embedded real-time software debugging , 2009, SAC '09.

[10]  Paolo Tonella,et al.  Analysis and testing of Web applications , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[11]  Jens Grabowski,et al.  A Model for Usage-Based Testing of Event-Driven Software , 2011, 2011 Fifth International Conference on Secure Software Integration and Reliability Improvement - Companion.

[12]  Mika Katara,et al.  Experiences of System-Level Model-Based GUI Testing of an Android Application , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[13]  Atif M. Memon,et al.  Developing a Single Model and Test Prioritization Strategies for Event-Driven Software , 2011, IEEE Transactions on Software Engineering.

[14]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[15]  Paolo Tonella,et al.  Combining model-based and combinatorial testing for effective test case generation , 2012, ISSTA 2012.

[16]  Atif M. Memon,et al.  Using transient/persistent errors to develop automated test oracles for event-driven software , 2004, Proceedings. 19th International Conference on Automated Software Engineering, 2004..

[17]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..