Testing & Verification In Service-Oriented Architecture : A Survey

Service-Oriented Architecture (SOA) is gaining momentum as an emerging distributed system architecture for business-to-business collaborations. This momentum can be observed in both industry and academic research. SOA presents new challenges and opportunities for testing and verification leading to an upsurge in research. This paper surveys the previous work undertaken on testing and verification of service-centric systems, which in total are 177 papers, showing the strengths and weaknesses of current strategies and testing tools and identifying issues for future work. Copyright c © 2009 John Wiley & Sons, Ltd.

[1]  S. Seshu,et al.  Introduction to the theory of finite-state machines , 1963 .

[2]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[3]  Darrell D. E. Long,et al.  Theory of finite automata with an introduction to formal languages , 1989 .

[4]  Amir Pnueli,et al.  Temporal Logic in Specification , 1987, Lecture Notes in Computer Science.

[5]  Tadao Murata,et al.  Petri nets: Properties, analysis and applications , 1989, Proc. IEEE.

[6]  Larry J Morell,et al.  A Theory of Fault-Based Testing , 1990, IEEE Trans. Software Eng..

[7]  Elaine J. Weyuker,et al.  Analyzing Partition Testing Strategies , 1991, IEEE Trans. Software Eng..

[8]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[9]  Gilbert Thomas Laycock,et al.  The theory and practice of specification based software testing , 1993 .

[10]  Gregg Rothermel,et al.  A safe, efficient regression test selection technique , 1997, TSEM.

[11]  Steven J. DeRose,et al.  XML Path Language (XPath) , 1999 .

[12]  Karsten Wolf,et al.  LoLA: A Low Level Analyser , 2000, ICATPN.

[13]  Stephan Merz,et al.  Model Checking , 2000 .

[14]  Alessandro Orso,et al.  Regression test selection for Java software , 2001, OOPSLA '01.

[15]  Ibrahim K. El-Far.  Enjoying the Perks of Model-Based Testing , 2001 .

[16]  Amit P. Sheth,et al.  Composition, Performance Analysis and Simulation of Web Services , 2002 .

[17]  Dong Wang,et al.  Extending WSDL to facilitate Web services testing , 2002, 7th IEEE International Symposium on High Assurance Systems Engineering, 2002. Proceedings..

[18]  Raymond A. Paul,et al.  Coyote: an XML-based framework for Web services testing , 2002, 7th IEEE International Symposium on High Assurance Systems Engineering, 2002. Proceedings..

[19]  Kurt Jensen.  Coloured Petri Nets: Status and Outlook , 2003, ICATPN.

[20]  Ana R. Cavalli,et al.  New approaches for passive testing using an Extended Finite State Machine specification , 2003, Inf. Softw. Technol..

[21]  Raymond A. Paul,et al.  Verification of Web services using an enhanced UDDI server , 2003, Proceedings of the Eighth International Workshop on Object-Oriented Real-Time Dependable Systems, 2003. (WORDS 2003)..

[22]  Clémentine Nebut,et al.  Requirements by contracts allow automated system testing , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[23]  H. Lan,et al.  SWRL : A semantic Web rule language combining OWL and ruleML , 2004 .

[24]  Chen Fu,et al.  Testing of java web services for robustness , 2004, ISSTA '04.

[25]  Raymond A. Paul,et al.  Cooperative and group testing in verification of dynamic composite Web services , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[26]  A. Jefferson Offutt,et al.  Generating test cases for web services using data perturbation , 2004, SOEN.

[27]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[28]  Jie Xu,et al.  Testing Web Services , 2004 .

[29]  Senthil Mani,et al.  WS-I Basic Profile: a practitioner's view , 2004, Proceedings. IEEE International Conference on Web Services, 2004..

[30]  Krys J. Kochut,et al.  A CP-nets-based design and verification framework for Web services composition , 2004, Proceedings. IEEE International Conference on Web Services, 2004..

[31]  Zongyuan Yang,et al.  Metamorphic Testing and Its Applications , 2004 .

[32]  Xiang Fu,et al.  Analysis of interacting BPEL web services , 2004, WWW '04.

[33]  Raymond A. Paul,et al.  Testing Web Services Using Progressive Group Testing , 2004, AWCC.

[34]  Ming Ye,et al.  Web services interoperability testing based on ontology , 2005, The Fifth International Conference on Computer and Information Technology (CIT'05).

[35]  Raymond A. Paul,et al.  Swiss Cheese Test Case Generation for Web Services Testing , 2005, IEICE Trans. Inf. Syst..

[36]  Reiko Heckel,et al.  Towards Contract-based Testing of Web Services , 2005, Electron. Notes Theor. Comput. Sci..

[37]  Juan Luo,et al.  Testing Web services by XML perturbation , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[38]  Wil M. P. van der Aalst,et al.  WofBPEL: A Tool for Automated Analysis of BPEL Processes , 2005, ICSOC.

[39]  Shing-Chi Cheung,et al.  Towards a metamorphic testing methodology for service-oriented software applications , 2005, Fifth International Conference on Quality Software (QSIC'05).

[40]  Wei-Tek Tsai,et al.  WSDL-based automatic test case generation for Web services testing , 2005, IEEE International Workshop on Service-Oriented System Engineering (SOSE'05).

[41]  Cesare Pautasso,et al.  The JOpera visual composition language , 2005, J. Vis. Lang. Comput..

[42]  Abbas Tarhini,et al.  A Simple Approach for Testing Web Service Based Applications , 2005, IICS.

[43]  Raymond A. Paul,et al.  Adaptive testing, oracle generation, and test case ranking for Web services , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[44]  Holger Schlingloff,et al.  Modeling and Model Checking Web Services , 2005, LCMAS.

[45]  Lu Zhang,et al.  A framework for testing Web services and its supporting tool , 2005, IEEE International Workshop on Service-Oriented System Engineering (SOSE'05).

[46]  Tevfik Bultan,et al.  Verifiable Web services with hierarchical interfaces , 2005, IEEE International Conference on Web Services (ICWS'05).

[47]  Cesare Pautasso,et al.  JOpera: an agile environment for Web service composition with visual unit testing and refactoring , 2005, 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC'05).

[48]  Raymond A. Paul,et al.  A robust testing framework for verifying Web services by completeness and consistency analysis , 2005, IEEE International Workshop on Service-Oriented System Engineering (SOSE'05).

[49]  Raymond A. Paul,et al.  Automated model checking and testing for composite Web services , 2005, Eighth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC'05).

[50]  Wei-Tek Tsai,et al.  Web service group testing with windowing mechanisms , 2005, IEEE International Workshop on Service-Oriented System Engineering (SOSE'05).

[51]  Jia Zhang,et al.  Criteria analysis and validation of the reliability of Web services-oriented systems , 2005, IEEE International Conference on Web Services (ICWS'05).

[52]  Wei Sun,et al.  BPEL4WS unit testing: framework and implementation , 2005, IEEE International Conference on Web Services (ICWS'05).

[53]  Wei-Tek Tsai,et al.  Voting multi-dimensional data with deviations for Web services under group testing , 2005, 25th IEEE International Conference on Distributed Computing Systems Workshops.

[54]  Dan Pilone,et al.  UML 2.0 in a Nutshell (In a Nutshell (O'Reilly)) , 2005 .

[55]  Reiko Heckel,et al.  Automatic Conformance Testing of Web Services , 2005, FASE.

[56]  Andrea Polini,et al.  The audition framework for testing Web services interoperability , 2005, 31st EUROMICRO Conference on Software Engineering and Advanced Applications.

[57]  Ying Jiang,et al.  Contract-based mutation for testing components , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[58]  Yong Xiao,et al.  Verifying web services composition based on hierarchical colored petri nets , 2005, IHIS '05.

[59]  David A. Carrington,et al.  A passive test oracle using a component's API , 2005, 12th Asia-Pacific Software Engineering Conference (APSEC'05).

[60]  Initial Investigations into Interoperability Testing of Web Services from their Specification using the Unified Modelling Language , 2006 .

[61]  Hong Zhu,et al.  A Framework for Service-Oriented Testing of Web Services , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[62]  Wei Sun,et al.  A Graph-Search Based Approach to BPEL4WS Test Generation , 2006, 2006 International Conference on Software Engineering Advances (ICSEA'06).

[63]  Shihong Huang,et al.  WSDLTest - A Tool for Testing Web Services , 2006, 2006 Eighth IEEE International Symposium on Web Site Evolution (WSE'06).

[64]  Claudio de la Riva,et al.  Generation of Conformance Test Suites for Compositions of Web Services Using Model Checking , 2006, Testing: Academic & Industrial Conference - Practice And Research Techniques (TAIC PART'06).

[65]  Kai Xu,et al.  Testing Scenario Implementation with Behavior Contracts , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[66]  Amit M. Paradkar,et al.  Model-based functional conformance testing of web services operating on persistent data , 2006, TAV-WEB '06.

[67]  Wei Sun,et al.  BPEL4WS Unit Testing: Test Case Generation Using a Concurrent Path Analysis Approach , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[68]  Arcot Sowmya,et al.  A Sniffer Based Approach to WS Protocols Conformance Checking , 2006, 2006 Fifth International Symposium on Parallel and Distributed Computing.

[69]  Philip Mayer,et al.  Towards a BPEL unit testing framework , 2006, TAV-WEB '06.

[70]  Gerardo Canfora,et al.  Testing services and service-centric systems: challenges and opportunities , 2006, IT Professional.

[71]  Silvia Regina Vergilio,et al.  Exploring Perturbation Based Testing for Web Services , 2006, 2006 IEEE International Conference on Web Services (ICWS'06).

[72]  Gwyduk Yeom,et al.  A QoS model and testing mechanism for quality-driven Web services selection , 2006, The Fourth IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems, and the Second International Workshop on Collaborative Computing, Integration, and Assurance (SEUS-WCCIA'06).

[73]  Niels Lohmann,et al.  Analyzing Interacting BPEL Processes , 2006, Business Process Management.

[74]  Lei Li,et al.  Performance engineering of service compositions , 2006, SOSE '06.

[75]  Feng Lin,et al.  Applying Safe Regression Test Selection Techniques to Java Web Services , 2006, International Conference on Next Generation Web Services Practices.

[76]  Abbas Tarhini,et al.  Regression Testing Web Services-based Applications , 2006, IEEE International Conference on Computer Systems and Applications, 2006..

[77]  Hang Yu,et al.  Testing BPEL-based Web Service Composition Using High-level Petri Nets , 2006, 2006 10th IEEE International Enterprise Distributed Object Computing Conference (EDOC'06).

[78]  Arcot Sowmya,et al.  ASEHA: A Framework for Modelling and Verification ofWeb Services Protocols , 2006, Fourth IEEE International Conference on Software Engineering and Formal Methods (SEFM'06).

[79]  Hang Yu,et al.  Web Service Testing Method Based on Fault-coverage , 2006, 2006 10th IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW'06).

[80]  Gerardo Canfora,et al.  SOA: Testing and Self-Checking , 2006 .

[81]  Colin Atkinson,et al.  Strategies for the Run-Time Testing of Third Party Web Services , 2007, IEEE International Conference on Service-Oriented Computing and Applications (SOCA '07).

[82]  Juan-Zi Li,et al.  Ontology-Based Test Case Generation for Testing Web Services , 2007, Eighth International Symposium on Autonomous Decentralized Systems (ISADS'07).

[83]  Xiaoying Bai,et al.  A Framework for Model Checking Web Service Compositions Based on BPEL4WS , 2007, IEEE International Conference on e-Business Engineering (ICEBE'07).

[84]  Andrea Polini,et al.  Automatic generation of test-beds for pre-deployment QoS evaluation of web services , 2007, WOSP '07.

[85]  Zhiwei Xu,et al.  An Approach to Debugging Grid or Web Services , 2007, IEEE International Conference on Web Services (ICWS 2007).

[86]  Raymond A. Paul,et al.  A Coverage Relationship Model for Test Case Selection and Ranking for Multi-version Software , 2007, 10th IEEE High Assurance Systems Engineering Symposium (HASE'07).

[87]  Tao Xie,et al.  Automated Testing and Response Analysis ofWeb Services , 2007, IEEE International Conference on Web Services (ICWS 2007).

[88]  Hernán C. Melgratti,et al.  Testing Service Composition , 2007 .

[89]  Michael Ruth,et al.  Concurrency Issues in Automating RTS for Web Services , 2007, IEEE International Conference on Web Services (ICWS 2007).

[90]  Xiaoying Bai,et al.  Contract-Based Testing for Web Services , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[91]  Wei-Tek Tsai,et al.  A Framework for Contract-Based Collaborative Verification and Validation of Web Services , 2007, CBSE.

[92]  Mark Grechanik,et al.  Automatic Test Generation From GUI Applications For Testing Web Services , 2007, 2007 IEEE International Conference on Software Maintenance.

[93]  Michael Ruth,et al.  Towards Automatic Regression Test Selection for Web Services , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[94]  Eda Marchetti,et al.  Automatic Test Data Generation for XML Schema-based Partition Testing , 2007, Second International Workshop on Automation of Software Test (AST '07).

[95]  Amit M. Paradkar,et al.  Automated Functional Conformance Test Generation for Semantic Web Services , 2007, IEEE International Conference on Web Services (ICWS 2007).

[96]  Mark Harman.  Automated Test Data Generation using Search Based Software Engineering , 2007, Second International Workshop on Automation of Software Test (AST '07).

[97]  Toru Yamaguchi,et al.  Interoperability verification for Web Service based robot communication platforms , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[98]  M. Vieira,et al.  Benchmarking the Robustness of Web Services , 2007, 13th Pacific Rim International Symposium on Dependable Computing (PRDC 2007).

[99]  Ana R. Cavalli,et al.  Timed Modeling of Web Services Composition for Automatic Testing , 2007, 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System.

[100]  Ying Yu,et al.  OWL-S Based Interaction Testing of Web Service-Based System , 2007, Third International Conference on Next Generation Web Services Practices (NWeSP'07).

[101]  Andrew Glover,et al.  Continuous Integration: Improving Software Quality and Reducing Risk (The Addison-Wesley Signature Series) , 2007 .

[102]  Gerardo Canfora,et al.  Search-based testing of service level agreements , 2007, GECCO '07.

[103]  Jun Zhang,et al.  A Business Process of Web Services Testing Method Based on UML2.0 Activity Diagram , 2007, Workshop on Intelligent Information Technology Application (IITA 2007).

[104]  Gerardo Canfora,et al.  Web Services Regression Testing , 2007, Test and Analysis of Web Services.

[105]  Mohamed Adel Serhani,et al.  New Approach for EFSM-Based Passive Testing of Web Services , 2007, TestCom/FATES.

[106]  T. S. E. Maibaum,et al.  An Approach to Integration Testing of Object-Oriented Programs , 2007, Seventh International Conference on Quality Software (QSIC 2007).

[107]  Yongyan Zheng,et al.  Analysis of BPEL Data Dependencies , 2007, 33rd EUROMICRO Conference on Software Engineering and Advanced Applications (EUROMICRO 2007).

[108]  Ruth Breu,et al.  Model Driven Testing of SOA – based Software , 2007 .

[109]  Yongyan Zheng,et al.  A Model Checking based Test Case Generation Framework forWeb Services , 2007, Fourth International Conference on Information Technology (ITNG'07).

[110]  Michael Ruth,et al.  A Safe Regression Test Selection Technique for Web Services , 2007, Second International Conference on Internet and Web Applications and Services (ICIW'07).

[111]  Jie Xu,et al.  Determining the dependability of Service-Oriented Architectures , 2007, Int. J. Simul. Process. Model..

[112]  Nuno Laranjeiro,et al.  wsrbench: An On-Line Tool for Robustness Benchmarking , 2008, 2008 IEEE International Conference on Services Computing.

[113]  A.T. Endo,et al.  Web Services Composition Testing: A Strategy Based on Structural Testing of Parallel Programs , 2008, Testing: Academic & Industrial Conference - Practice and Research Techniques (taic part 2008).

[114]  Ana R. Cavalli,et al.  Automatic Timed Test Case Generation for Web Services Composition , 2008, 2008 Sixth European Conference on Web Services.

[115]  Colin Atkinson,et al.  Specifying High-Assurance Services , 2008, Computer.

[116]  T. H. Tse,et al.  Data flow testing of service-oriented workflow applications , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[117]  Rui Wang,et al.  Requirement Model-Based Mutation Testing for Web Service , 2008, 2008 4th International Conference on Next Generation Web Services Practices.

[118]  Liam Peyton,et al.  Integration Testing of Composite Applications , 2008, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[119]  Thomas Wallet,et al.  On-The-Fly Model-Based Testing of Web Services with Jambition , 2008, WS-FM.

[120]  Hanpin Wang,et al.  Modeling and verifying BPEL using synchronized net , 2008, SAC '08.

[121]  Hong Zhu,et al.  Ontology for Service Oriented Testing of Web Services , 2008, 2008 IEEE International Symposium on Service-Oriented System Engineering.

[122]  Gerardo Canfora,et al.  Service-Oriented Architectures Testing: A Survey , 2009, ISSSE.

[123]  Tao Zhang,et al.  Testing BPEL with Stream X-Machine , 2008, 2008 International Symposium on Information Science and Engineering.

[124]  Andrea Polini,et al.  Model-Based Generation of Testbeds for Web Services , 2008, TestCom/FATES.

[125]  M. Driss,et al.  QoS testing of service-based applications , 2008, 2008 3rd International Design and Test Workshop.

[126]  Wu Chou,et al.  Control Flow Analysis and Coverage Driven Testing for Web Services , 2008, 2008 IEEE International Conference on Web Services.

[127]  Shoichi Morimoto,et al.  A Survey of Formal Verification for Business Process Modeling , 2008, ICCS.

[128]  Yinong Chen,et al.  Automatic Mutation Testing and Simulation on OWL-S Specified Web Services , 2008, 41st Annual Simulation Symposium (anss-41 2008).

[129]  Tao Zhang,et al.  WSDL-Based Automated Test Data Generation for Web Service , 2008, 2008 International Conference on Computer Science and Software Engineering.

[130]  Bixin Li,et al.  Regression Testing of Composite Service: An XBFG-Based Approach , 2008, 2008 IEEE Congress on Services Part II (services-2 2008).

[131]  Tao Xie,et al.  Quota-constrained test-case prioritization for regression testing of service-centric systems , 2008, 2008 IEEE International Conference on Software Maintenance.

[132]  Malcolm Munro,et al.  Fault-Based Web Services Testing , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[133]  Jun Zhu,et al.  Surrogate: A Simulation Apparatus for Continuous Integration Testing in Service Oriented Architecture , 2008, 2008 IEEE International Conference on Services Computing.

[134]  Wei-Tek Tsai,et al.  Ontology-Based Test Modeling and Partition Testing of Web Services , 2008, 2008 IEEE International Conference on Web Services.

[135]  Eda Marchetti,et al.  WS-TAXI: A WSDL-based Testing Tool for Web Services , 2009, 2009 International Conference on Software Testing Verification and Validation.

[136]  Zhiwei Xu,et al.  A Model of Message-Based Debugging Facilities for Web or Grid Services , 2009, 2009 Congress on Services - I.

[137]  Nigel Shadbolt,et al.  Resource Description Framework (RDF) , 2009 .

[138]  Rui Wang,et al.  Mutation Test Based on OWL-S Requirement Model , 2009, 2009 IEEE International Conference on Web Services.

[139]  Prashant Kumar,et al.  Test-the-REST: An Approach to Testing RESTful Web-Services , 2009, 2009 Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns.

[140]  Ina Schieferdecker,et al.  Applying Model Checking to Generate Model-Based Integration Tests from Choreography Models , 2009, TestCom/FATES.

[141]  Hao Wang,et al.  FLTL-MC: Online High Level Program Analysis for Web Services , 2009, 2009 Congress on Services - I.

[142]  Mark Harman,et al.  Regression Testing Minimisation, Selection and Prioritisation - A Survey , 2009 .

[143]  Fabrice Bouquet,et al.  Using Common Criteria to Assess Quality of Web Services , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[144]  Nawal Guermouche,et al.  Timed Model Checking Based Approach for Web Services Analysis , 2009, 2009 IEEE International Conference on Web Services.

[145]  Tien-Dung Cao,et al.  Testing Web Services Composition Using the TGSE Tool , 2009, 2009 Congress on Services - I.

[146]  Inmaculada Medina-Bulo,et al.  GAmera: An Automatic Mutant Generation System for WS-BPEL Compositions , 2009, 2009 Seventh IEEE European Conference on Web Services.

[147]  Nuno Laranjeiro,et al.  Improving Web Services Robustness , 2009, 2009 IEEE International Conference on Web Services.

[148]  T. H. Tse,et al.  Tag-Based Techniques for Black-Box Test Case Prioritization for Service Testing , 2009, 2009 Ninth International Conference on Quality Software.

[149]  Walid Gaaloul,et al.  Semantic Web Services for Satisfying SOA Requirements , 2008, Advances in Web Semantics I.

[150]  T. H. Tse,et al.  Data flow testing of service choreography , 2009, ESEC/FSE '09.

[151]  Eda Marchetti,et al.  Whitening SOA testing , 2009, ESEC/FSE '09.

[152]  Wu Chou,et al.  An Abstract GFSM Model for Optimal and Incremental Conformance Testing of Web Services , 2009, 2009 IEEE International Conference on Web Services.

[153]  Wu Chou,et al.  A Combinatorial Approach to Multi-session Testing of Stateful Web Services , 2009, 2009 Congress on Services - I.

[154]  Jun Zhu,et al.  A Unified Test Framework for Continuous Integration Testing of SOA Solutions , 2009, 2009 IEEE International Conference on Web Services.

[155]  Yujia Ge,et al.  Search-Based Performance Testing of Applications with Composite Services , 2009, 2009 International Conference on Web Information Systems and Mining.

[156]  Saurabh Sinha,et al.  Efficient Testing of Service-Oriented Applications Using Semantic Service Stubs , 2009, 2009 IEEE International Conference on Web Services.

[157]  Taratip Suwannasart,et al.  An Improved Test Case Generation Method for Web Service Testing from WSDL-S and OCL with Pair-Wise Testing Technique , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[158]  Jing Sun,et al.  A Framework for Testing Semantic Web Services Using Model Checking , 2009, 2009 Fourth South-East European Workshop on Formal Methods.

[159]  Bellanov S. Apilli.  Fault-based combinatorial testing of web services , 2009, OOPSLA Companion.

[160]  Lu Zhang,et al.  Generating Effective Test Sequences for BPEL Testing , 2009, 2009 Ninth International Conference on Quality Software.

[161]  Xiang Li,et al.  SOArMetrics: A Toolkit for Testing and Evaluating SOA Middleware , 2009, 2009 Congress on Services - I.

[162]  Jun Zhu,et al.  Towards a practical and effective method for Web services test case generation , 2009, 2009 ICSE Workshop on Automation of Software Test.

[163]  Fatiha Zaïdi,et al.  A Formal Framework for Service Orchestration Testing Based on Symbolic Transition Systems , 2009, TestCom/FATES.

[164]  Byungjeong Lee,et al.  Automatic Discovery of Web Services Based on Dynamic Black-Box Testing , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[165]  Raquel Blanco,et al.  A First Approach to Test Case Generation for BPEL Compositions of Web Services Using Scatter Search , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[166]  Sebastian Wieczorek,et al.  MBT4Chor: A Model-Based Testing Approach for Service Choreographies , 2009, ECMDA-FA.

[167]  T. H. Tse,et al.  Test case prioritization for regression testing of service-oriented business applications , 2009, WWW '09.

[168]  Baowen Xu,et al.  Test Case Prioritization for Web Service Regression Testing , 2010, 2010 Fifth IEEE International Symposium on Service Oriented System Engineering.

[169]  Fuyuki Ishikawa,et al.  Business Semantics Centric Reliability Testing for Web Services in BPEL , 2010, 2010 6th World Congress on Services.

[170]  Marc-Florian Wendland,et al.  Using the UML Testing Profile for Enterprise Service Choreographies , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[171]  Javier Tuya,et al.  Testing Long-Lived Web Services Transactions Using a Risk-Based Approach , 2010, 2010 10th International Conference on Quality Software.

[172]  Fevzi Belli,et al.  Event-driven modeling and testing of real-time web services , 2010, Service Oriented Computing and Applications.

[173]  Yunzhan Gong,et al.  Defect Analysis Respecting Dead Path Elimination in BPEL Process , 2010, 2010 IEEE Asia-Pacific Services Computing Conference.

[174]  Schahram Dustdar,et al.  Script-Based Generation of Dynamic Testbeds for SOA , 2010, 2010 IEEE International Conference on Web Services.

[175]  Dong Qiu,et al.  Automatic test case selection and generation for regression testing of composite service based on extensible BPEL flow graph , 2010, 2010 IEEE International Conference on Software Maintenance.

[176]  Wei Fu,et al.  Automatic Testing of Web Services in Haskell Platform , 2010 .

[177]  Valentin Pavlov,et al.  A Composable Framework for Test Automation of Service-Based Applications , 2010, 2010 Seventh International Conference on the Quality of Information and Communications Technology.

[178]  Philip Samuel,et al.  Web services regression test case prioritization , 2010, 2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM).

[179]  Yunzhan Gong,et al.  A static analysis method of WSDL related defect pattern in BPEL , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[180]  Andrea Polini,et al.  A Counter-Example Testing Approach for Orchestrated Services , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[181]  Sébastien Salva,et al.  A Preliminary Study on BPEL Process Testability , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[182]  Hassan Reza,et al.  A Framework for Testing RESTful Web Services , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[183]  Xiang Li,et al.  Towards a Formal Verification Approach for Business Process Coordination , 2010, 2010 IEEE International Conference on Web Services.

[184]  Schahram Dustdar,et al.  Testbeds for Emulating Dependability Issues of Mobile Web Services , 2010, 2010 6th World Congress on Services.

[185]  Ana R. Cavalli,et al.  WebMov: A Dedicated Framework for the Modelling and Testing of Web Services Composition , 2010, 2010 IEEE International Conference on Web Services.

[186]  Javier Tuya,et al.  A Proactive Approach to Test Service Level Agreements , 2010, 2010 Fifth International Conference on Software Engineering Advances.

[187]  Tien-Dung Cao,et al.  Automated Runtime Verification for Web Services , 2010, 2010 IEEE International Conference on Web Services.

[188]  Ana R. Cavalli,et al.  A Fault Injection Tool for Testing Web Services Composition , 2010, TAIC PART.

[189]  Colin Atkinson,et al.  Testing Web-Services Using Test Sheets , 2010, 2010 Fifth International Conference on Software Engineering Advances.

[190]  Manuel Palomo-Duarte,et al.  Takuan: A Tool for WS-BPEL Composition Testing Using Dynamic Invariant Generation , 2010, ICWE.

[191]  Chao Liu,et al.  A Regression Testing Method for Composite Web Service , 2010, 2010 International Conference on Biomedical Engineering and Computer Science.

[192]  Ruth Breu,et al.  A Tool-Based Methodology for System Testing of Service-Oriented Systems , 2010, 2010 Second International Conference on Advances in System Testing and Validation Lifecycle.

[193]  Lei Zhou,et al.  Automatically Testing Web Services Choreography with Assertions , 2010, ICFEM.

[194]  Tien-Dung Cao,et al.  Online Testing Framework for Web Services , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[195]  Márcio Eduardo Delamaro,et al.  Built-In Structural Testing of Web Services , 2010, 2010 Brazilian Symposium on Software Engineering.

[196]  Lionel C. Briand,et al.  A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation , 2010, IEEE Transactions on Software Engineering.

[197]  Nuno Laranjeiro,et al.  Applying Text Classification Algorithms in Web Services Robustness Testing , 2010, 2010 29th IEEE Symposium on Reliable Distributed Systems.

[198]  A. Askarunisa,et al.  A test case reduction method for semantic based web services , 2010, 2010 Second International conference on Computing, Communication and Networking Technologies.

[199]  Jin Liu,et al.  OCL-Based Testing for E-Learning Web Service , 2010, ICWL Workshops.

[200]  Ana R. Cavalli,et al.  Testing Web Service Orchestrators in Context: A Symbolic Approach , 2010, 2010 8th IEEE International Conference on Software Engineering and Formal Methods.

[201]  Ana R. Cavalli,et al.  Timed Extended Invariants for the Passive Testing of Web Services , 2010, 2010 IEEE International Conference on Web Services.

[202]  Kera Z. Watkins,et al.  Introducing fault-based combinatorial testing to web services , 2010, Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon).

[203]  T. H. Tse,et al.  Taking Advantage of Service Selection: A Study on the Testing of Location-Based Web Services Through Test Case Prioritization , 2010, 2010 IEEE International Conference on Web Services.

[204]  Simone do Rocio Senger de Souza,et al.  Event- and Coverage-Based Testing of Web Services , 2010, 2010 Fourth International Conference on Secure Software Integration and Reliability Improvement Companion.

[205]  Sébastien Salva,et al.  Stateful Web Service Robustness , 2010, 2010 Fifth International Conference on Internet and Web Applications and Services.

[206]  Manuel Núñez,et al.  Passive Testing of Web Services , 2010, WS-FM.

[207]  Iman Saleh,et al.  Formal Specification and Verification of Data-Centric Service Composition , 2010, 2010 IEEE International Conference on Web Services.

[208]  Mark Harman,et al.  Automatically generating realistic test input from web services , 2011, Proceedings of 2011 IEEE 6th International Symposium on Service Oriented System (SOSE).