Early fault detection with model-based testing

Current and future trends for software include increasingly complex requirements on interaction between systems. As a result, the difficulty of system testing increases. Model-based testing is a test technique where test cases are generated from a model of the system. In this study we explore model-based testing on the system level, starting from early development. We apply model-based testing to a subsystem of a message gateway product in order to improve early fault detection. The results are compared to another subsystem that is tested with hand-crafted test cases. Based on our experiences, we present a set of challenges and recommendations for system-level, model-based testing. Our results indicate that model-based testing, starting from early development, significantly increases the number of faults detected during system testing.

[1]  Lars Mathiassen,et al.  Successful process implementation , 2004, IEEE Software.

[2]  Barry Boehm,et al.  Top 10 list [software development] , 2001 .

[3]  Alexander Pretschner,et al.  One evaluation of model-based testing and its automation , 2005, ICSE.

[4]  Bruno Legeard,et al.  A taxonomy of model-based testing , 2006 .

[5]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[6]  Yuri Gurevich,et al.  Evolving algebras 1993: Lipari guide , 1995, Specification and validation methods.

[7]  Michael R. Lowry,et al.  Combining test case generation and runtime verification , 2005, Theor. Comput. Sci..

[8]  Adrian M. Colyer.  From research to reward: challenges in technology transfer , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[9]  Margus Veanes,et al.  Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer , 2008, Formal Methods and Testing.

[10]  Troels Mørch.  Test Driven Development , 2005 .

[11]  Egon Börger,et al.  High Level System Design and Analysis Using Abstract State Machines , 1998, FM-Trends.

[12]  Mark Blackburn,et al.  Why Model-Based Test Automation is Different and What You Should Know to Get Started , 2004 .

[13]  Alexander Pretschner,et al.  Model-Based Testing in Practice , 2005, FM.

[14]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[15]  Yuri Gurevich,et al.  Sequential abstract-state machines capture sequential algorithms , 2000, TOCL.

[16]  Miguel P Caldas,et al.  Research design: qualitative, quantitative, and mixed methods approaches , 2003 .

[17]  Lars-Ola Damm,et al.  Early and Cost-Effective Software Fault Detection: Measurement and Implementation in an Industrial Setting , 2007 .

[18]  Didar Zowghi,et al.  A Maturity Model for the Implementation of Software Process Improvement: an Empirical Study , 2022 .

[19]  John Hughes,et al.  Testing telecoms software with quviq QuickCheck , 2006, ERLANG '06.

[20]  Thierry Heuillard,et al.  AGEDIS Case Studies: Model-Based Testing in Industry , 2003 .

[21]  Alexander Pretschner,et al.  Abstractions for Model-Based Testing , 2005, Electron. Notes Theor. Comput. Sci..

[22]  Ian J. Hayes,et al.  Specification directed module testing , 1986, IEEE Transactions on Software Engineering.

[23]  Barry W. Boehm,et al.  Some future trends and implications for systems and software engineering processes , 2006, Syst. Eng..

[24]  Barry W. Boehm,et al.  Software Defect Reduction Top 10 List , 2001, Computer.

[25]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[26]  Joe Armstrong,et al.  Programming Erlang: Software for a Concurrent World , 1993 .

[27]  Alan Bundy,et al.  Constructing Induction Rules for Deductive Synthesis Proofs , 2006, CLASE.