Automatic Grammar-Based Test Generation

In this paper, we present an automatic grammar-based test generation approach which takes a symbolic grammar as input, requires zero control input from users, and produces well-distributed test cases. Our approach utilizes a novel dynamic stochastic model where each variable is associated with a tuple of probability distributions, which are dynamically adjusted along the derivation. The adjustment is based on a tabling strategy to keep track of the recursion of each grammar variable. We further present a test generation coverage tree illustrating the distribution of generated test cases and their detailed derivations, more importantly, it provides various implicit balance control mechanisms. We implemented this approach in a Java-based system, named Gena. Experimental results demonstrate the effectiveness of our test generation approach and show the balanced distribution of generated test cases over grammatical structures.

[1]  Dragan Gasevic,et al.  Grammar-based test generation for software product line feature models , 2012, CASCON.

[2]  Emin Gün Sirer,et al.  Using production grammars in software testing , 1999, DSL '99.

[3]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[4]  W. M. McKeeman,et al.  Differential Testing for Software , 1998, Digit. Tech. J..

[5]  Alexander Pretschner,et al.  One evaluation of model-based testing and its automation , 2005, ICSE.

[6]  Yu Lei,et al.  A Test Generation Strategy for Pairwise Testing , 2002, IEEE Trans. Software Eng..

[7]  K. V. Hanford,et al.  Automatic Generation of Test Cases , 1970, IBM Syst. J..

[8]  Katalin Tarnay,et al.  Testing of Communicating Systems , 1999, IFIP — The International Federation for Information Processing.

[9]  Peter M. Maurer,et al.  The design and implementation of a grammar‐based data generator , 1992, Softw. Pract. Exp..

[10]  Peter M. Maurer,et al.  Generating test data with enhanced context-free grammars , 1990, IEEE Software.

[11]  David Scott Warren,et al.  Memoing for logic programs , 1992, CACM.

[12]  Perdita Stevens,et al.  Modelling Recursive Calls with UML State Diagrams , 2003, FASE.

[13]  Ralf Lämmel,et al.  Controllable Combinatorial Coverage in Grammar-Based Testing , 2006, TestCom.

[14]  Rupak Majumdar,et al.  Directed test generation using symbolic grammars , 2007, ASE.

[15]  Nicolas Halbwachs,et al.  Automatic testing of reactive systems , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[16]  Ralf Lämmel,et al.  Grammar Testing , 2001, FASE.

[17]  Jirapun Daengdej,et al.  A Test Case Generation Process and Technique , 2010 .

[18]  Fevzi Belli,et al.  Model-based testing of web service compositions , 2011, Proceedings of 2011 IEEE 6th International Symposium on Service Oriented System (SOSE).

[19]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[20]  Paul A. Strooper,et al.  Grammar‐based test generation with YouGen , 2011, Softw. Pract. Exp..

[21]  Adam Kiezun,et al.  Grammar-based whitebox fuzzing , 2008, PLDI '08.

[22]  Gopal Gupta,et al.  Simplifying dynamic programming via mode‐directed tabling , 2008, Softw. Pract. Exp..