Generating boundary values from OCL constraints using constraints rewriting and search algorithms

A key component of model-based testing is the generation of test data from constraints (e.g., specified in the Object Constraint Language (OCL)) associated with models e.g., specified in the Unified Modeling Language (UML). The quality of test data eventually determines the effectiveness of test cases, e.g., fault detection and coverage. A simple way to generate test data from an OCL constraint is to find a set of values for all the variables that satisfies the constraint. One way of improving the quality of test data is by generating test data at the boundaries of each variable in the constraint. In this paper, we extend our search-based test data generation approach to generate test data at the boundaries of each variable involved in a constraint. We present different cases of constraints involving different types of variables and how we rewrite them in order to automatically generate boundary values using our existing OCL Solver called EsOCL based on a set of heuristics. We empirically evaluated our heuristics using three search algorithms, i.e., Genetic Algorithm, (1+1) Evolutionary Algorithm, and Alternating Variable Method, on industrial OCL constraints from our previous work. Results show that AVM on average achieved a 95% of success rate and outperformed the rest of the algorithms in finding all the boundary values of constraints in fewer generations.

[1]  Andrea Arcuri,et al.  Improved heuristics for solving OCL constraints using search algorithms , 2014, GECCO.

[2]  Manuel Clavel,et al.  Checking Unsatisfiability for OCL Constraints , 2010, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[3]  Lionel C. Briand,et al.  Modeling robustness behavior using aspect-oriented modeling to support robustness testing of industrial systems , 2011, Software & Systems Modeling.

[4]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[5]  Behzad Bordbar,et al.  UML2ALLOY: A tool for lightweight modelling of discrete event systems , 2005, IADIS AC.

[6]  Gabriele Taentzer,et al.  Translation of Restricted OCL Constraints into Graph Constraints for Generating Meta Model Instances by Graph Grammars , 2008, GT-VMT@ETAPS.

[7]  Lionel C. Briand,et al.  Generating Test Data from OCL Constraints with Search Techniques , 2013, IEEE Transactions on Software Engineering.

[8]  Mark Harman Search Based Software Engineering , 2006, International Conference on Computational Science.

[9]  Lionel C. Briand,et al.  A practical guide for using statistical tests to assess randomized algorithms in software engineering , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[10]  Phil McMinn,et al.  Search-based software test data generation: a survey: Research Articles , 2004 .

[11]  Alexander Knapp,et al.  Executing Underspecified OCL Operation Contracts with a SAT Solver , 2008, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[12]  Daniel Jackson,et al.  Alcoa: the Alloy constraint analyzer , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[13]  Li Qing,et al.  Test Case Automate Generation from UML Sequence Diagram and OCL Expression , 2007 .

[14]  Yuanyuan Zhang,et al.  Search Based Software Engineering: A Comprehensive Analysis and Review of Trends Techniques and Applications , 2009 .

[15]  Gordon Fraser,et al.  On Parameter Tuning in Search Based Software Engineering , 2011, SSBSE.

[16]  Jordi Cabot Sagrera,et al.  Verification of UML/OCL Class Diagrams Using Constraint Programming , 2008 .

[17]  Yuanyuan Zhang,et al.  Search-based software engineering: Trends, techniques and applications , 2012, CSUR.

[18]  Lionel C. Briand,et al.  Empirical investigation of search algorithms for environment model-based testing of real-time embedded software , 2012, ISSTA 2012.

[19]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[20]  Achim D. Brucker,et al.  A Specification-Based Test Case Generation Method for UML/OCL , 2010, MoDELS.

[21]  Stephan Weißleder,et al.  Deriving Input Partitions from UML Models for Automatic Test Generation , 2008, MoDELS.

[22]  Joost-Pieter Katoen,et al.  Towards Model Checking OCL , 2000 .

[23]  Bogdan Korel,et al.  Automated Software Test Data Generation , 1990, IEEE Trans. Software Eng..

[24]  L. C. Briand,et al.  Model Transformations as a Strategy to Automate Model-Based Testing - A Tool and Industrial Case Studies, Version 1.0 , 2010 .

[25]  Lee J. White,et al.  A Domain Strategy for Computer Program Testing , 1980, IEEE Transactions on Software Engineering.

[26]  Andrea Arcuri,et al.  It really does matter how you normalize the branch distance in search‐based software testing , 2013, Softw. Test. Verification Reliab..

[27]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[28]  Lori A. Clarke,et al.  A Close Look at Domain Testing , 1982, IEEE Transactions on Software Engineering.

[29]  Shaukat Ali,et al.  Evaluating Normalization Functions with Search Algorithms for Solving OCL Constraints , 2014, ICTSS.

[30]  Thomas Jensen,et al.  UML-CASTING: Test synthesis from UML models using constraint resolution1 , 2002 .

[31]  Lionel C. Briand,et al.  A Search-Based OCL Constraint Solver for Model-Based Test Data Generation , 2011, 2011 11th International Conference on Quality Software.

[32]  Lionel C. Briand,et al.  A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation , 2010, IEEE Transactions on Software Engineering.