Criteria for generating specification-based tests

This paper presents general criteria for generating test inputs from state-based specifications. Software testing can only be formalized and quantified when a solid basis for test generation can be defined. Formal specifications of complex systems represent a significant opportunity for testing because they precisely describe what functions the software is supposed to provide in a form that can easily be manipulated. These techniques provide coverage criteria that are based on the specifications, and are made up of several parts, including test prefixes that contain inputs necessary to put the software into the appropriate state for the test values. The test generation process includes several steps for transforming specifications to tests. Empirical results from a comparative case study application of these criteria are presented.

[1]  Kathryn L. Heninger.  Specifying Software Requirements for Complex Systems: New Techniques and Their Application , 2001, IEEE Transactions on Software Engineering.

[2]  John Gannon,et al.  State-based model checking of event-driven system requirements , 1991, SIGSOFT '91.

[3]  Wei-Tek Tsai,et al.  Automated Test Case Generation for Programs Specified by Relational Algebra Queries , 1990, IEEE Trans. Software Eng..

[4]  Elaine J. Weyuker,et al.  An Applicable Family of Data Flow Testing Criteria , 1988, IEEE Trans. Software Eng..

[5]  Marie-Claude Gaudel,et al.  Software testing based on formal specifications: a theory and a tool , 1991, Softw. Eng. J..

[6]  Paul Ammann,et al.  Using formal methods to derive test frames in category-partition testing , 1994, Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance.

[7]  Thomas J. Ostrand,et al.  Automatic generation of test scripts from formal test specifications , 1989 .

[8]  Rudolph E. Seviora,et al.  Automatic failure detection with Conditional-Belief supervisors , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[9]  Rudolph E. Seviora,et al.  An approach to automatic detection of software failures , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[10]  Phyllis G. Frankl,et al.  Case studies on testing object-oriented programs , 1991, TAV4.

[11]  J. R. Horgan,et al.  A data flow coverage testing tool for C , 1992, [1992] Proceedings of the Second Symposium on Assessment of Quality Software Development Tools.

[12]  Constance L. Heitmeyer,et al.  Tools for formal specification, verification, and validation of requirements , 1997, Proceedings of COMPASS '97: 12th Annual Conference on Computer Assurance.

[13]  Luqi,et al.  Constructing an automated testing oracle: an effort to produce reliable software , 1994, Proceedings Eighteenth Annual International Computer Software and Applications Conference (COMPSAC 94).

[14]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[15]  David A. Carrington,et al.  A Framework for Specification-Based Testing , 1996, IEEE Trans. Software Eng..

[16]  A. Jefferson Offutt,et al.  SOFL: A Formal Engineering Methodology for Industrial Applications , 1998, IEEE Trans. Software Eng..

[17]  Ian J. Hayes,et al.  Specification directed module testing , 1986, IEEE Transactions on Software Engineering.

[18]  Mark Blackburn,et al.  T-VEC: a tool for developing critical systems , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.

[19]  Zhenyi Jin.  Deriving mode invariants from SCR specifications , 1996, Proceedings of ICECCS '96: 2nd IEEE International Conference on Engineering of Complex Computer Systems (held jointly with 6th CSESAW and 4th IEEE RTAW).

[20]  Thomas J. Ostrand,et al.  Automatic Generation of Test Scripts from Formal Test Specifications , 1989, Symposium on Testing, Analysis, and Verification.

[21]  Robert M. Hierons,et al.  Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[22]  A. Jefferson Offutt,et al.  Generating Tests from UML Specifications , 1999, UML.

[23]  Richard A. Kemmerer,et al.  Testing Formal Specifications to Detect Design Errors , 1985, IEEE Transactions on Software Engineering.

[24]  Alexandre Petrenko,et al.  Test Selection Based on Communicating Nondeterministic Finite-State Machines Using a Generalized WP-Method , 1994, IEEE Trans. Software Eng..

[25]  Sriram Sankar,et al.  Structural specification-based testing with ADL , 1996, ISSTA '96.

[26]  Elaine J. Weyuker,et al.  Automatically Generating Test Data from a Boolean Specification , 1994, IEEE Trans. Software Eng..

[27]  Robert M. Hierons.  Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[28]  Gilbert Thomas Laycock,et al.  Formal specification and testing: A case study , 1992, Softw. Test. Verification Reliab..

[29]  Laurent Fribourg,et al.  Test sets generation from algebraic specifications using logic programming , 1986, J. Syst. Softw..

[30]  Shaoying Liu,et al.  Generating test data from SOFL specifications , 1999, J. Syst. Softw..

[31]  Richard G. Hamlet,et al.  Data Abstraction, Implementation, Specification, and Testing , 1981, TOPL.

[32]  Steven P. Miller,et al.  Applicability of modified condition/decision coverage to software testing , 1994, Softw. Eng. J..