Developing the Basic Verification Action (BVA) Structure Towards Test Oracle Automation

This paper proposes the Basic Verification Action (BVA) structure to verify outcomes produced during and at the end of test cases execution. BVA items are determined, stored and maintained as external entities to the actual test cases, forming a metadata repository comprised of a combination of technical and business knowledge of the software application. When applied to the final outcomes of the test case, the verification items actually represent the test oracle. Although the proposed model may not address all types of test oracles, it is posited that there is a significant number of cases where it is applicable. In these cases, applying the proposed mechanism can lead to more efficient testing, eventually driving test oracle automation. Benefits and shortcomings of using BVAs are discussed.

[1]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[2]  Sarfraz Khurshid,et al.  Software assurance by bounded exhaustive testing , 2004, IEEE Transactions on Software Engineering.

[3]  Robert V. Binder,et al.  Design for testability in object-oriented systems , 1994, CACM.

[4]  Douglas Hoffman.  Using Test Oracles in Automation , 2003 .

[5]  John C. Cherniavsky,et al.  Validation, Verification, and Testing of Computer Software , 1982, CSUR.

[6]  Erik Agrell,et al.  Automatic testing of StreamBits , 2007 .

[7]  Standard Glossary of Terms used in Software Testing Version 3 . 1 Foundation Terms , .

[8]  James A. Whittaker,et al.  Model‐Based Software Testing , 2002 .

[9]  A. C. Coulter.  Graybox software testing methodology: embedded software testing technique , 1999, Gateway to the New Millennium. 18th Digital Avionics Systems Conference. Proceedings (Cat. No.99CH37033).

[10]  Barry W. Boehm,et al.  Software development cost estimation approaches — A survey , 2000, Ann. Softw. Eng..

[11]  Michael R. Lowry,et al.  Combining test case generation and runtime verification , 2005, Theor. Comput. Sci..

[12]  Phyllis G. Frankl,et al.  The ASTOOT approach to testing object-oriented programs , 1994, TSEM.

[13]  Tsipi Heart,et al.  What Is a Test Case? Revisiting the Software Test Case Concept , 2009, EuroSPI.

[14]  Abraham Kandel,et al.  Using a neural network in the software testing process , 2002, Int. J. Intell. Syst..

[15]  David Coppit,et al.  The Galileo fault tree analysis tool , 1999, Digest of Papers. Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing (Cat. No.99CB36352).

[16]  Maher Lamari,et al.  Towards an automated test generation for the verification of model transformations , 2007, SAC '07.

[17]  T.C. Lethbridge,et al.  Guide to the Software Engineering Body of Knowledge (SWEBOK) and the Software Engineering Education Knowledge (SEEK) - a preliminary mapping , 2001, 10th International Workshop on Software Technology and Engineering Practice.