A robust testing framework for verifying Web services by completeness and consistency analysis

This paper presents a specification based robust testing framework for Web services. Web services testing is done by 1) extracting condition and event combinations from the Web service specification; 2) ensuring that these combinations are consistent with each other; 3) performing completeness analysis to identify all the missing condition and event combinations; 4) identifying the locations where updates are needed to maintain the completeness and consistency; and 5) emphasizing on robustness testing by generating positive as well as negative test cases. An efficient algorithm called Covering Scenario Generation is proposed to identify the locations where incompleteness and inconsistency exist. The algorithm is based on the min-terms of Boolean expressions that combine multiple conditions into a single checkable item. The proposed algorithm has been experimented with several large industrial applications and the results indicated that the proposed algorithm is robust and scalable to large applications. A case study is designed to illustrate the design and testing process.

[1]  Nashat Mansour,et al.  Testing Web services , 2005, The 3rd ACS/IEEE International Conference onComputer Systems and Applications, 2005..

[2]  Raymond A. Paul,et al.  Adaptive testing, oracle generation, and test case ranking for Web services , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[3]  Constance L. Heitmeyer,et al.  Automated consistency checking of requirements specifications , 1996, TSEM.

[4]  Gerard J. Holzmann,et al.  The SPIN Model Checker , 2003 .

[5]  Stephan Merz,et al.  Model Checking , 2000 .

[6]  Jie Xu,et al.  Testing Web Services , 2004 .

[7]  Raymond A. Paul,et al.  Swiss Cheese Test Case Generation for Web Services Testing , 2005, IEICE Trans. Inf. Syst..

[8]  Thomas A. Henzinger,et al.  Generating tests from counterexamples , 2004, Proceedings. 26th International Conference on Software Engineering.

[9]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[10]  Wei-Tek Tsai,et al.  Rapid Pattern-Oriented Scenario-Based Testing for Embedded Systems , 2005 .

[11]  Masao Mukaidono,et al.  Independence of each axiom in a set of axioms and complete sets of axioms of Boolean algebra , 2002, Proceedings 32nd IEEE International Symposium on Multiple-Valued Logic.

[12]  Kenneth H. Rosen,et al.  Discrete Mathematics and its applications , 2000 .

[13]  Raymond A. Paul,et al.  Developing and assuring trustworthy Web services , 2005, Proceedings Autonomous Decentralized Systems, 2005. ISADS 2005..

[14]  B.C.H. Turton.  Extending Quine-McCluskey for Exclusive-Or logic synthesis , 1996 .

[15]  D. W. Cooper.  Adaptive testing , 1976, ICSE '76.

[16]  Gianfranco R. Putzolu,et al.  Statistical Complexity of Algorithms for Boolean Function Minimization , 1965, JACM.

[17]  Thomas A. Henzinger,et al.  Lazy abstraction , 2002, POPL '02.

[18]  Tsong Yueh Chen,et al.  Test case selection strategies based on Boolean specifications ‡ , 2001, Softw. Test. Verification Reliab..

[19]  Masao Mukaidono,et al.  Independence of the axioms of Boolean algebra in multiple-valued logic , 2000, Proceedings 30th IEEE International Symposium on Multiple-Valued Logic (ISMVL 2000).