A taxonomy of model-based testing

Model-based testing relies on models of a system under test and/or its environment to derive test cases for the system. This paper provides an overview of the field. Seven different dimensions define a taxonomy that allows the characterization of different approaches to model-based testing. It is intended to help with understanding benefits and limitations of model-based testing, understanding the approach used in a particular model-based testing tool, and understanding the issues involved in integrating model-based testing into a software development process. To illustrate the taxonomy, we classify several approaches embedded in existing model-based testing tools.

[1]  Duncan Clarke,et al.  Automated Test and Oracle Generation for Smart-Card Applications , 2001, E-smart.

[2]  Jim Clarke.  Automated test generation from a behavioral model , 1998 .

[3]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[4]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[5]  Stacy J. Prowell,et al.  JUMBL: a tool for model-based statistical testing , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[6]  Christian Schallhart,et al.  Tools for Test Case Generation , 2004, Model-Based Testing of Reactive Systems.

[7]  Nicolae Goga,et al.  Formal Test Automation: A Simple Experiment , 1999, IWTCS.

[8]  Bruno Legeard,et al.  Preamble computation in automated test case generation using constraint logic programming , 2004, Softw. Test. Verification Reliab..

[9]  Simeon C. Ntafos,et al.  A Comparison of Some Structural Testing Strategies , 1988, IEEE Trans. Software Eng..

[10]  Daniel Geist,et al.  Semi-formal test generation with Genevieve , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[11]  Duncan Clarke,et al.  STG: A Symbolic Test Generation Tool , 2002, TACAS.

[12]  Raimund Kirner,et al.  Real-Time and Hybrid Systems Testing , 2004, Model-Based Testing of Reactive Systems.

[13]  Insup Lee,et al.  A Temporal Logic Based Theory of Test Coverage and Generation , 2002, TACAS.

[14]  Laurent Fournier,et al.  Developing an architecture validation suite: application to the PowerPC architecture , 1999, DAC '99.

[15]  Bruno Marre,et al.  Test sequences generation from LUSTRE descriptions: GATEL , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[16]  A. Pretschner.  Classical search strategies for test case generation with Constraint Logic Programming , 2001 .

[17]  Alexander Pretschner,et al.  Model-Based Testing in Practice , 2005, FM.

[18]  Alexander Pretschner,et al.  One evaluation of model-based testing and its automation , 2005, ICSE.

[19]  Elaine J. Weyuker,et al.  An Applicable Family of Data Flow Testing Criteria , 1988, IEEE Trans. Software Eng..

[20]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[21]  Axel Belinfante,et al.  Automated Testing in Practice: The Highway Tolling System , 2002, TestCom.

[22]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[23]  Mark Blackburn,et al.  Why Model-Based Test Automation is Different and What You Should Know to Get Started , 2004 .

[24]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[25]  Alexander Pretschner,et al.  Model Based Testing for Real: The Inhouse Card Case Study , 2001 .

[26]  Fabrice Bouquet,et al.  Mastering Test Generation from Smart Card Software Formal Models , 2004, CASSIS.

[27]  César Viho,et al.  Formal Test Automation: The Conference Protocol with TGV/TORX , 2000, TestCom.

[28]  Jesse H. Poore,et al.  Generating transition probabilities to support model‐based software testing , 2000, Softw. Pract. Exp..

[29]  W. Gutjahr.  Partition Testing versus Random Testing: the Innuence of Uncertainty , 1999 .

[30]  Thomas Santen,et al.  Automating Test Case Generation from Z Specifications with Isabelle , 1997, ZUM.

[31]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[32]  Jesse H. Poore,et al.  Generating transition probabilities to support model‐based software testing , 2000 .

[33]  Marko Samer,et al.  Technology of Test-Case Generation , 2004, Model-Based Testing of Reactive Systems.

[34]  Alexander Pretschner,et al.  Model-Based Test Case Generation for Smart Cards , 2003, FMICS.

[35]  Alfred V. Aho,et al.  An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours , 1991, IEEE Trans. Commun..

[36]  Axel van Lamsweerde,et al.  Formal specification: a roadmap , 2000, ICSE '00.

[37]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[38]  Alexander Pretschner,et al.  Methodological Issues in Model-Based Testing , 2004, Model-Based Testing of Reactive Systems.

[39]  Dieter Hogrefe,et al.  Autolink-a tool for automatic test generation from SDL specifications , 1998, Proceedings. 2nd IEEE Workshop on Industrial Strength Formal Specification Techniques.

[40]  Hendrik Brinksma,et al.  Côte de Resyste : Automated Model Based Testing , 2002 .

[41]  Thierry Heuillard,et al.  AGEDIS Case Studies: Model-Based Testing in Industry , 2003 .

[42]  Jian Shen,et al.  An RTL Abstraction Technique for Processor Microarchitecture Validation and Test Generation , 2000, J. Electron. Test..

[43]  Alan Hartman,et al.  Using a model-based test generator to test for standard conformance , 2002, IBM Syst. J..

[44]  Christophe Gaston,et al.  Evaluating Coverage Based Testing , 2004, Model-Based Testing of Reactive Systems.

[45]  Vijayan N. Nair,et al.  A statistical assessment of some software testing strategies and application of experimental design techniques , 1998 .

[46]  Mohammad El-Ramly,et al.  Case Studies , 2004, Model-Based Testing of Reactive Systems.

[47]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[48]  Bruno Legeard,et al.  Generation of test sequences from formal specifications: GSM 11‐11 standard case study , 2004, Softw. Pract. Exp..

[49]  César Viho,et al.  An Industrial Experiment in Automatic Generation of Executable Test Suites for a Cache Coherency Protocol , 1998, IWTCS.

[50]  Alexander Pretschner,et al.  Abstractions for Model-Based Testing , 2005, Electron. Notes Theor. Comput. Sci..

[51]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[52]  Walter J. Gutjahr,et al.  Partition Testing vs. Random Testing: The Influence of Uncertainty , 1999, IEEE Trans. Software Eng..

[53]  John W. Horch.  Black-box testing: Techniques for functional testing of software and systems: by Boris BEIZER, John Wiley and Sons; New York, USA; 1995; xxvi + 294 pp.; $29.95. ISBN: 0-471-12094-4 , 1996 .

[54]  Thierry Jéron,et al.  TGV : theory , principles and algorithms A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2004 .

[55]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[56]  D. Rayner.  OSI Conformance Testing , 1987, Comput. Networks.

[57]  Claude Caci,et al.  Testing object-oriented systems , 2000, SOEN.

[58]  Richard G. Hamlet,et al.  Partition Testing Does Not Inspire Confidence , 1990, IEEE Trans. Software Eng..

[59]  Mladen A. Vouk.  Software Reliability Engineering , 1999 .

[60]  Amit M. Paradkar.  Case studies on fault detection effectiveness of model based test generation techniques , 2005, A-MOST.

[61]  Bruno Legeard,et al.  Boundary coverage criteria for test generation from formal models , 2004, 15th International Symposium on Software Reliability Engineering.