Grammar-Based Testing Using Realistic Domains in PHP

This paper presents an integration of grammar-based testing in a framework for contract-based testing in PHP. It relies on the notion of \gtypes, that make it possible to assign domains to data, by means of contract assertions written inside the source code of a PHP application. Then a test generation tool uses the contracts to generate relevant test data for unit testing. Finally a runtime assertion checker validates the assertions inside the contracts (among others membership of data to \gtypes) to establish the conformance verdict. We introduce here the possibility to generate and validate complex textual data specified by a grammar written in a dedicated grammar description language. This approach is tool-supported and experimented on the validation of web applications.

[1]  Bertrand Meyer.  PROGRAMMING FOR REUSABILITY AND EXTENDIBILITY , 1987 .

[2]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[3]  Cyril S. Ku,et al.  Design Patterns , 2008, Wiley Encyclopedia of Computer Science and Engineering.

[4]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[5]  Peter Thiemann,et al.  Contract-Driven Testing of JavaScript Code , 2010, TOOLS.

[6]  Daniel J. Rosenkrantz,et al.  Properties of deterministic top down grammars , 1969, STOC.

[7]  Ralf Lämmel,et al.  Controllable Combinatorial Coverage in Grammar-Based Testing , 2006, TestCom.

[8]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[9]  Colin J. Ihrig.  JavaScript Object Notation , 2013 .

[10]  Pieter J. Mosterman,et al.  Model-Based Testing for Embedded Systems , 2011, Computational Analysis, Synthesis, & Design Dynamic Systems.

[11]  K. Rustan M. Leino,et al.  The Spec# Programming System: An Overview , 2004, CASSIS.

[12]  Fabrice Bouquet,et al.  Selective Test Generation Method for Evolving Critical Systems , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[13]  Philippe Flajolet,et al.  A Calculus for the Random Generation of Labelled Combinatorial Structures , 1994, Theor. Comput. Sci..

[14]  J. Offutt,et al.  Mutation testing implements grammar-based testing , 2006, Second Workshop on Mutation Analysis (Mutation 2006 - ISSRE Workshops 2006).

[15]  Sarfraz Khurshid,et al.  TestEra: a novel framework for automated testing of Java programs , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[16]  Bernhard K. Aichernig,et al.  Contract-Based Testing , 2002, 10th Anniversary Colloquium of UNU/IIST.

[17]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[18]  Bertrand Meyer,et al.  Eiffel: programming for reusability and extendibility , 1987, SIGP.

[19]  Alain Giorgetti,et al.  Praspel: A Specification Language for Contract-Based Testing in PHP , 2011, ICTSS.

[20]  David Coppit,et al.  yagg: an easy-to-use generator for structured test inputs , 2005, ASE.

[21]  Hong-Yi Wang,et al.  Grammar Based Testing of HTML Injection Vulnerabilities in RSS Feeds , 2009, 2009 Testing: Academic and Industrial Conference - Practice and Research Techniques.

[22]  Sarfraz Khurshid,et al.  Test generation through programming in UDITA , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[23]  Gary T. Leavens,et al.  A Simple and Practical Approach to Unit Testing: The JML and JUnit Way , 2002, ECOOP.

[24]  Peter M. Maurer,et al.  Generating test data with enhanced context-free grammars , 1990, IEEE Software.

[25]  Bernhard K. Aichernig,et al.  Formal Methods at the Crossroads. From Panacea to Foundational Support , 2003, Lecture Notes in Computer Science.

[26]  Sarfraz Khurshid,et al.  Software assurance by bounded exhaustive testing , 2004, IEEE Transactions on Software Engineering.

[27]  Paul A. Strooper,et al.  Grammar‐based test generation with YouGen , 2011, Softw. Pract. Exp..

[28]  Adam Kiezun,et al.  Grammar-based whitebox fuzzing , 2008, PLDI '08.

[29]  Frank Tip,et al.  Finding bugs in dynamic web applications , 2008, ISSTA '08.

[30]  Per Madsen,et al.  Unit Testing Using Design by Contract and Equivalence Partitions , 2003, XP.

[31]  Mark Hennessy,et al.  Analysing the effectiveness of rule-coverage as a reduction criterion for test suites of grammar-based software , 2008, Empirical Software Engineering.

[32]  Albert L. Baker,et al.  JML: A Notation for Detailed Design , 1999, Behavioral Specifications of Businesses and Systems.

[33]  William E. Howden,et al.  Functional program testing and analysis , 1986 .

[34]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[35]  Christopher G. Lasater,et al.  Design Patterns , 2008, Wiley Encyclopedia of Computer Science and Engineering.