Combining model-based and risk-based testing for effective test case generation

Model-based testing employs models of the system under test to automatically generate test cases. In this paper, we propose an iterative approach, in which these models are refined based on the principles of risk-based testing. We use Markov Chains as system models, in which transitions among system states are annotated with probabilities. Initially, these probability values are equal and as such, states have equal chances for being visited by the generated test cases. Memory leaks are monitored during the execution of these test cases. Then, transition probabilities are updated based on the risk that a failure can occur due to the observed memory leaks. We applied our approach in the context of an industrial case study for model-based testing of a Smart TV system. We observed promising results, in which several crash failures were detected after an iteration of model refinement. We aim at automating the whole process based on an adaptation model using the history of recorded memory leaks during previous test executions.

[1]  Hasan Sözer,et al.  Improving Models for Model-Based Testing Based on Exploratory Testing , 2014, 2014 IEEE 38th International Computer Software and Applications Conference Workshops.

[2]  Alexander Pretschner,et al.  Model-Based Testing in Practice , 2005, FM.

[3]  Jesse H. Poore,et al.  Generating transition probabilities to support model‐based software testing , 2000, Softw. Pract. Exp..

[4]  Andreas Hoffmann,et al.  Model-Based Testing , 2012, IEEE Software.

[5]  Harry Robinson F INITE STATE MODEL-BASED TESTING ON A SHOESTRING , 1999 .

[6]  James A. Whittaker,et al.  A Markov Chain Model for Statistical Software Testing , 1994, IEEE Trans. Software Eng..

[7]  Thomas Thelin,et al.  Practical experiences with statistical usage testing , 2003, Eleventh Annual International Workshop on Software Technology and Engineering Practice.

[8]  Rajesh Subramanyan,et al.  A survey on model-based testing approaches: a systematic review , 2007, WEASELTech '07.

[9]  Pablo César,et al.  System software for digital television applications , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[10]  Richard Torkar,et al.  Software fault prediction metrics: A systematic literature review , 2013, Inf. Softw. Technol..

[11]  Frank Böhr Model Based Statistical Testing and Durations , 2010, 2010 17th IEEE International Conference and Workshops on Engineering of Computer Based Systems.

[12]  Mika Mäntylä,et al.  The Role of the Tester's Knowledge in Exploratory Software Testing , 2013, IEEE Transactions on Software Engineering.

[13]  Jonas Boberg Early fault detection with model-based testing , 2008, ERLANG '08.

[14]  Frank Böhr Model Based Statistical Testing of Embedded Systems , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[15]  R. Eschbach,et al.  From Requirements to Statistical Testing of Embedded Systems , 2007, Fourth International Workshop on Software Engineering for Automotive Systems (SEAS '07).

[16]  Ajay Chander,et al.  Optimal Test Input Sequence Generation for Finite State Models and Pushdown Systems , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.