Implementing Conformiq Qtronic

Conformiq Qtronic is a commercial tool for model driven testing. It derives tests automatically from behavioral system models. These are black-box tests [1] by nature, which means that they depend on the model and the interfaces of the system under test, but not on the internal structure (e.g. source code) of the implementation. In this essay, which accompanies my invited talk, I survey the nature of Conformiq Qtronic, the main implementation challenges that we have encountered and how we have approached them.

[1]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[2]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[3]  Krzysztof R. Apt,et al.  Principles of constraint programming , 2003 .

[4]  Flemming Nielson,et al.  Principles of Program Analysis , 1999, Springer Berlin Heidelberg.

[5]  Stefan Jähnichen.  Compiler construction : 8th international conference, CC '99, held as part of the Joint European Conferences on Theory and Practice of Software, ETAPS'99, Amsterdam, The Netherlands, March 22-28, 1999 : proceedings , 1999 .

[6]  K. Larsen,et al.  Online Testing of Real-time Systems Using Uppaal , 2004, FATES.

[7]  Gareth Lee,et al.  Using symbolic execution to guide test generation: Research Articles , 2005 .

[8]  Thomas W. Reps,et al.  Program Specialization via Program Slicing , 1996, Dagstuhl Seminar on Partial Evaluation.

[9]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[10]  Nicolae Goga,et al.  Probabilities In The Torx Test Derivation Algorithm , 2000, International Conference on Security and Management.

[11]  Arnaud Gotlieb,et al.  Automatic test data generation using constraint solving techniques , 1998, ISSTA '98.

[12]  R. Nigel Horspool,et al.  Faster Generalized LR Parsing , 1999, CC.

[13]  Ajitha Rajan,et al.  Requirements Coverage as an Adequacy Measure for Conformance Testing , 2008, ICFEM.

[14]  Kimmo Nupponen,et al.  The Design and Implementation of a Graph Rewrite Engine for Model Transformations , 2005 .

[15]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[16]  Ian J. Hayes,et al.  FM 2005: Formal Methods, International Symposium of Formal Methods Europe, Newcastle, UK, July 18-22, 2005, Proceedings , 2005, FM.

[17]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[18]  Nina Yevtushenko,et al.  Testing Transition Systems with Input and Output Testers , 2003, TestCom.

[19]  Joost Engelfriet,et al.  Node Replacement Graph Grammars , 1997, Handbook of Graph Grammars.

[20]  Grzegorz Rozenberg,et al.  Handbook of Graph Grammars and Computing by Graph Transformations, Volume 1: Foundations , 1997 .

[21]  Alfred V. Aho,et al.  Deterministic parsing of ambiguous grammars , 1973, POPL.

[22]  Frank Budinsky,et al.  Eclipse Modeling Framework , 2003 .

[23]  Robin Milner,et al.  A Theory of Type Polymorphism in Programming , 1978, J. Comput. Syst. Sci..

[24]  A. Jefferson Offutt,et al.  An Empirical Evaluation of Weak Mutation , 1994, IEEE Trans. Software Eng..

[25]  Ed Brinksma,et al.  Testing Real-Time Multi Input-Output Systems , 2005, ICFEM.

[26]  Richard J. Lipton,et al.  Theoretical and empirical studies on using program mutation to test the functional correctness of programs , 1980, POPL '80.

[27]  Mark Davis,et al.  The Unicode Standard, Version 3.0 , 2000 .

[28]  Bran Selic.  UML 2: A model-driven development tool , 2006, IBM Syst. J..

[29]  Alexandre Petrenko,et al.  Test Selection Based on Communicating Nondeterministic Finite-State Machines Using a Generalized WP-Method , 1994, IEEE Trans. Software Eng..

[30]  Robert D. Tennent,et al.  Semantics of programming languages , 1991, Prentice Hall International Series in Computer Science.

[31]  Benjamin C. Pierce,et al.  Types and programming languages: the next generation , 2003, 18th Annual IEEE Symposium of Logic in Computer Science, 2003. Proceedings..

[32]  Nicolas Halbwachs,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2005, Lecture Notes in Computer Science.

[33]  Alexandre Petrenko,et al.  Selecting test sequences for partially-specified nondeterministic finite state machines , 1995 .

[34]  Jan Tretmans,et al.  A Formal Approach to Conformance Testing , 1993, Protocol Test Systems.

[35]  Carl A. Gunter.  Semantics of programming languages: structures and techniques , 1993, Choice Reviews Online.

[36]  Gordon D. Plotkin,et al.  A structural approach to operational semantics , 2004, J. Log. Algebraic Methods Program..

[37]  Henrik C. Bohnenkamp,et al.  Timed Testing with TorX , 2005, FM.

[38]  Katalin Tarnay,et al.  Testing of Communicating Systems , 1999, IFIP — The International Federation for Information Processing.

[39]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[40]  Sarfraz Khurshid,et al.  Generalized Symbolic Execution for Model Checking and Testing , 2003, TACAS.

[41]  José C. Cunha,et al.  Grid Computing: Software Environments and Tools , 2005 .

[42]  Jon Postel,et al.  User Datagram Protocol , 1980, RFC.

[43]  Paul Ammann,et al.  Abstracting formal specifications to generate software tests via model checking , 1999, Gateway to the New Millennium. 18th Digital Avionics Systems Conference. Proceedings (Cat. No.99CH37033).

[44]  Marius Bozga,et al.  Using Static Analysis to Improve Automatic Test Generation , 2000, TACAS.

[45]  Patrick Cousot,et al.  Abstract interpretation , 1996, CSUR.

[46]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[47]  Hartmut Ehrig,et al.  Handbook of graph grammars and computing by graph transformation: vol. 3: concurrency, parallelism, and distribution , 1999 .

[48]  Axel Uhl,et al.  MDA Distilled , 2004 .

[49]  Keijo Heljanko,et al.  Specification coverage aided test selection , 2003, Third International Conference on Application of Concurrency to System Design, 2003. Proceedings..

[50]  John Morris,et al.  Using symbolic execution to guide test generation , 2005, Softw. Test. Verification Reliab..

[51]  Stefan P. Jaskiel,et al.  Systematic Software Testing , 2002 .

[52]  Bengt Jonsson,et al.  Specifying and Generating Test Cases Using Observer Automata , 2004, FATES.

[53]  Rina Dechter,et al.  Constraint Processing , 1995, Lecture Notes in Computer Science.

[54]  Guy L. Steele,et al.  The evolution of Lisp , 1993, HOPL-II.

[55]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[56]  James Gosling,et al.  The Java Language Specification, 3rd Edition , 2005 .

[57]  Guy L. Steele,et al.  Java(TM) Language Specification, The (3rd Edition) (Java (Addison-Wesley)) , 2005 .

[58]  K R.,et al.  Revised 5 . 97 Report on the Algorithmic Language Scheme — Rationale — , 2007 .

[59]  Brian Nielsen,et al.  Formal Approaches to Software Testing, 4th International Workshop, FATES 2004, Linz, Austria, September 21, 2004, Revised Selected Papers , 2005, FATES.

[60]  Richard Gabriel,et al.  The Evolution of Lisp , 2008, OOPSLA 2008.

[61]  Margus Veanes,et al.  Online testing with model programs , 2005, ESEC/FSE-13.

[62]  Rafael Dueire Lins,et al.  Garbage collection: algorithms for automatic dynamic memory management , 1996 .

[63]  Annegret Habel,et al.  Hyperedge Replacement, Graph Grammars , 1997, Handbook of Graph Grammars.