Formal Testing Approaches for Service-Oriented Architectures and Web Services: a Systematic Review

Service-oriented architectures and web services have been used as important technologies to foster the development of loosely coupled and distributed applications. Web services pose new challenges for the testing activity, mainly because mission-critical and complex business process systems are implemented with them. In this context, for- mal testing approaches are necessary to guarantee the service quality. In this document, we identify formal approaches proposed to test service-oriented architectures and web services, analyzing the application context, models, empirical study, authorship, and ref- erences. A systematic review was planned and executed to select the studies considered in this work. We observed that, though more experiments and cooperation are necessary, there is a considerable interest in the topic.

[1]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[2]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[3]  T. H. Tse,et al.  Test case prioritization for regression testing of service-oriented business applications , 2009, WWW '09.

[4]  Raymond A. Paul,et al.  Developing and assuring trustworthy Web services , 2005, Proceedings Autonomous Decentralized Systems, 2005. ISADS 2005..

[5]  Thomas Wallet,et al.  On-The-Fly Model-Based Testing of Web Services with Jambition , 2008, WS-FM.

[6]  Gerardo Canfora,et al.  Service-Oriented Architectures Testing: A Survey , 2009, ISSSE.

[7]  Byungjeong Lee,et al.  Automatic Discovery of Web Services Based on Dynamic Black-Box Testing , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[8]  Raymond A. Paul,et al.  A robust testing framework for verifying Web services by completeness and consistency analysis , 2005, IEEE International Workshop on Service-Oriented System Engineering (SOSE'05).

[9]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[10]  T. D. Fletcher,et al.  Web Services Choreography Description Language Version 1.0, W3C , 2004 .

[11]  Andrea Polini,et al.  Audition of Web Services for Testing Conformance to Open Specified Protocols , 2004, Architecting Systems with Trustworthy Components.

[12]  Mohamed Adel Serhani,et al.  Towards the Testing of Composed Web Services in 3rd Generation Networks , 2006, TestCom.

[13]  Chien-Hung Liu,et al.  A WS-BPEL Based Structural Testing Approach for Web Service Compositions , 2008, 2008 IEEE International Symposium on Service-Oriented System Engineering.

[14]  Alfred V. Aho,et al.  An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours , 1991, IEEE Trans. Commun..

[15]  Andrea Polini,et al.  Model-Based Generation of Testbeds for Web Services , 2008, TestCom/FATES.

[16]  Fevzi Belli,et al.  Event-Driven Modeling and Testing of Web Services , 2008, 2008 32nd Annual IEEE International Computer Software and Applications Conference.

[17]  Wu Chou,et al.  Control Flow Analysis and Coverage Driven Testing for Web Services , 2008, 2008 IEEE International Conference on Web Services.

[18]  Matthew B. Dwyer,et al.  Bogor: an extensible and highly-modular software model checking framework , 2003, ESEC/FSE-11.

[19]  Paula Gomes Mian,et al.  Systematic Review in Software Engineering , 2005 .

[20]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007 .

[21]  G. J. Tretmans Testing labelled Transition Systems with Inputs and Outputs , 1995 .

[22]  Reiko Heckel,et al.  A Model-Driven Approach to Discovery, Testing and Monitoring of Web Services , 2007, Test and Analysis of Web Services.

[23]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[24]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[25]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[26]  Tao Zhang,et al.  WSDL-Based Automated Test Data Generation for Web Service , 2008, 2008 International Conference on Computer Science and Software Engineering.

[27]  Amit M. Paradkar,et al.  Automated Functional Conformance Test Generation for Semantic Web Services , 2007, IEEE International Conference on Web Services (ICWS 2007).

[28]  Rance Cleaveland,et al.  Using formal specifications to support testing , 2009, CSUR.

[29]  Lionel C. Briand A Critical Analysis of Empirical Research in Software Testing , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[30]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[31]  A.T. Endo,et al.  Web Services Composition Testing: A Strategy Based on Structural Testing of Parallel Programs , 2008, Testing: Academic & Industrial Conference - Practice and Research Techniques (taic part 2008).

[32]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[33]  W. Chan,et al.  A Metamorphic Testing Approach for Online Testing of Service-Oriented Software Applications , 2007, Int. J. Web Serv. Res..

[34]  Wei Sun,et al.  BPEL4WS Unit Testing: Test Case Generation Using a Concurrent Path Analysis Approach , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[35]  T. H. Tse,et al.  An Empirical Study of the Use of Frankl-Weyuker Data Flow Testing Criteria to Test BPEL Web Services , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[36]  T. H. Tse,et al.  Data flow testing of service-oriented workflow applications , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[37]  Michael Ruth,et al.  Towards Automatic Regression Test Selection for Web Services , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[38]  Raymond A. Paul,et al.  Swiss Cheese Test Case Generation for Web Services Testing , 2005, IEICE Trans. Inf. Syst..

[39]  Hernán C. Melgratti,et al.  Testing Service Composition , 2007 .

[40]  Simone do Rocio Senger de Souza,et al.  Structural testing criteria for message‐passing parallel programs , 2008, Concurr. Comput. Pract. Exp..

[41]  T. H. Tse,et al.  Data flow testing of service choreography , 2009, ESEC/FSE '09.

[42]  Francisco Curbera,et al.  Web Services Business Process Execution Language Version 2.0 , 2007 .

[43]  Andrea Polini,et al.  Improving Trust in Composite eServices Via Run-Time Participants Testing , 2008, EGOV.

[44]  Sungwon Kang,et al.  Generating Test Cases for Web Services Using Extended Finite State Machine , 2006, TestCom.

[45]  Amy Nicole Langville,et al.  Google's PageRank and beyond - the science of search engine rankings , 2006 .

[46]  Reiko Heckel,et al.  Automatic Conformance Testing of Web Services , 2005, FASE.

[47]  Yongyan Zheng,et al.  An Automatic Test Case Generation Framework for Web Services , 2007, J. Softw..

[48]  Dimitris Dranidis,et al.  Formal Verification of Web Service Behavioural Conformance through Testing , 2007 .

[49]  Jun Zhu,et al.  Business-process-driven gray-box SOA testing , 2008, IBM Syst. J..

[50]  Ana R. Cavalli,et al.  Automatic Timed Test Case Generation for Web Services Composition , 2008, 2008 Sixth European Conference on Web Services.

[51]  Silvia Regina Vergilio,et al.  Structural testing criteria for message-passing parallel programs , 2008 .

[52]  Alberto Martelli,et al.  Verifying the Conformance of Web Services to Global Interaction Protocols: A First Step , 2005, EPEW/WS-FM.

[53]  Mike P. Papazoglou,et al.  Service oriented architectures: approaches, technologies and research issues , 2007, The VLDB Journal.

[54]  Hassan Artail,et al.  An Abstract Workflow-Based Framework for Testing Composed Web Services , 2007, 2007 IEEE/ACS International Conference on Computer Systems and Applications.

[55]  Dimitris Dranidis,et al.  Leveraging Semantic Web Service Descriptions for Validation by Automated Functional Testing , 2009, ESWC.