Automatic test case generation for WS-Agreements using combinatorial testing

In the scope of the applications developed under the service-based paradigm, Service Level Agreements (SLAs) are a standard mechanism used to flexibly specify the Quality of Service (QoS) that must be delivered. These agreements contain the conditions negotiated between the service provider and consumers as well as the potential penalties derived from the violation of such conditions. In this context, it is important to assure that the service based application (SBA) behaves as expected in order to avoid potential consequences like penalties or dissatisfaction between the stakeholders that have negotiated and signed the SLA. In this article we address the testing of SLAs specified using the WS-Agreement standard by means of applying testing techniques such as the Classification Tree Method and Combinatorial Testing to generate test cases. From the content of the individual terms of the SLA, we identify situations that need to be tested. We also obtain a set of constraints based on the SLA specification and the behavior of the SBA in order to guarantee the testability of the test cases. Furthermore, we define three different coverage strategies with the aim at grading the intensity of the tests. Finally, we have developed a tool named SLACT (SLA Combinatorial Testing) in order to automate the process and we have applied the whole approach to an eHealth case study. We propose a method to test SLAs using standard testing techniques.Constraints that avoid generating non feasible test cases are automatically obtained.Different coverage strategies are defined to grade the thoroughness of the tests.A tool named SLACT that automates the generation of the test cases is implemented.Application of the approach to a real and critical eHealth service-based scenario.

[1]  Thomas Wallet,et al.  On-The-Fly Model-Based Testing of Web Services with Jambition , 2008, WS-FM.

[2]  Luciano Baresi,et al.  Emerging Research Themes in Services-Oriented Systems , 2012, 2012 Annual SRII Global Conference.

[3]  Hartmut Ritter,et al.  A concept for QoS integration in Web services , 2003, Fourth International Conference on Web Information Systems Engineering Workshops, 2003. Proceedings..

[4]  Rodziah Atan,et al.  HA-SLA: A Hierarchical Autonomic SLA Model for SLA Monitoring in Cloud Computing , 2013 .

[5]  Paola Inverardi,et al.  Context-Aware Adaptive Services: The PLASTIC Approach , 2009, FASE.

[6]  James D. McCaffrey,et al.  An Empirical Study of Pairwise Test Set Generation Using a Genetic Algorithm , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[7]  Frank Leymann,et al.  Cost-based prevention of violations of service level agreements in composed services using self-adaptation , 2012, 2012 First International Workshop on European Software Services and Systems Research - Results and Challenges (S-Cube).

[8]  Yaron Goland,et al.  Web Services Business Process Execution Language , 2009, Encyclopedia of Database Systems.

[9]  George Spanoudakis,et al.  Monitoring WS-Agreement s: An Event Calculus-Based Approach , 2007, Test and Analysis of Web Services.

[10]  Andrea Polini,et al.  Model-Based Generation of Testbeds for Web Services , 2008, TestCom/FATES.

[11]  Asit Dan,et al.  Web services agreement specification (ws-agreement) , 2004 .

[12]  Asir S Vedamuthu,et al.  Web Services Policy 1.5 - Framework , 2007 .

[13]  Nuel D. Belnap,et al.  A Useful Four-Valued Logic , 1977 .

[14]  Dimitris Dranidis,et al.  SLAs for cross-layer adaptation and monitoring of service-based applications: a case study , 2011, QASBA '11.

[15]  Wolfgang Ziegler,et al.  Research Challenges in Managing and Using Service Level Agreements , 2009, CoreGRID@Euro-Par.

[16]  Marcos Palacios.  Defining an SLA-Aware Method to Test Service-Oriented Systems , 2011, ICSOC Workshops.

[17]  G. H. Gessert,et al.  Four valued logic for relational database systems , 1990, SGMD.

[18]  Wolfgang Emmerich,et al.  Service-Level Agreements for Electronic Services , 2010, IEEE Transactions on Software Engineering.

[19]  Paolo Traverso,et al.  Service-Oriented Computing: State of the Art and Research Challenges , 2007, Computer.

[20]  Javier Tuya,et al.  Identifying Test Requirements by Analyzing SLA Guarantee Terms , 2012, 2012 IEEE 19th International Conference on Web Services.

[21]  Javier Tuya,et al.  A Proactive Approach to Test Service Level Agreements , 2010, 2010 Fifth International Conference on Software Engineering Advances.

[22]  George Spanoudakis,et al.  Runtime Prediction of Software Service Availability , 2011 .

[23]  R. K. Shyamasundar,et al.  SLA Monitor: A System for Dynamic Monitoring of Adaptive Web Services , 2011, 2011 IEEE Ninth European Conference on Web Services.

[24]  Vladimir Tosic,et al.  Management applications of the Web Service Offerings Language (WSOL) , 2005, Inf. Syst..

[25]  Heiko Ludwig,et al.  The WSLA Framework: Specifying and Monitoring Service Level Agreements for Web Services , 2003, Journal of Network and Systems Management.

[26]  Paola Inverardi,et al.  A Framework for Analyzing and Testing the Performance of Software Services , 2008, ISoLA.

[27]  George Spanoudakis,et al.  Establishing and Monitoring SLAs in Complex Service Based Systems , 2009, 2009 IEEE International Conference on Web Services.

[28]  Marcos Palacios,et al.  Evaluating the Service Level Agreements of NDT under WS-Agreement - An Empirical Analysis , 2012, WEBIST.

[29]  Schahram Dustdar,et al.  Monitoring, Prediction and Prevention of SLA Violations in Composite Services , 2010, 2010 IEEE International Conference on Web Services.

[30]  E. F. Codd,et al.  The Relational Model for Database Management, Version 2 , 1990 .

[31]  Jos J. M. Trienekens,et al.  Specification of Service Level Agreements: Problems, Principles and Practices , 2004, Software Quality Journal.

[32]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[33]  Wolfgang Emmerich,et al.  Efficient online monitoring of web-service SLAs , 2008, SIGSOFT '08/FSE-16.

[34]  Ramin Yahyapour,et al.  Service Level Agreements for Cloud Computing , 2011 .

[35]  Gerardo Canfora,et al.  Search-based testing of service level agreements , 2007, GECCO '07.

[36]  Andrea Polini,et al.  SOA Test Governance: Enabling Service Integration Testing across Organization and Technology Borders , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[37]  Xavier Franch,et al.  SALMonADA: A platform for monitoring and explaining violations of WS-agreement-compliant documents , 2012, 2012 4th International Workshop on Principles of Engineering Service-Oriented Systems (PESOS).

[38]  Carlo Ghezzi,et al.  A journey to highly dynamic, self-adaptive service-based applications , 2008, Automated Software Engineering.

[39]  Ramin Yahyapour,et al.  Modeling Service Level Agreements with Binary Decision Diagrams , 2009, ICSOC/ServiceWave.

[40]  Gerardo Canfora,et al.  Testing services and service-centric systems: challenges and opportunities , 2006, IT Professional.

[41]  A. Jefferson Offutt,et al.  Using abstraction and Web applications to teach criteria-based test design , 2011, 2011 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T).

[42]  Wolfgang Emmerich,et al.  SLAng: a language for defining service level agreements , 2003, The Ninth IEEE Workshop on Future Trends of Distributed Computing Systems, 2003. FTDCS 2003. Proceedings..

[43]  Javier Tuya,et al.  Full predicate coverage for testing SQL database queries , 2010, Softw. Test. Verification Reliab..

[44]  Karim Djemame,et al.  Enabling service-level agreement renegotiation through extending WS-Agreement specification , 2014, Service Oriented Computing and Applications.

[45]  Javier Tuya,et al.  Coverage-Based Testing for Service Level Agreements , 2015, IEEE Transactions on Services Computing.

[46]  Xavier Franch,et al.  Comprehensive Explanation of SLA Violations at Runtime , 2014, IEEE Transactions on Services Computing.

[47]  Eric Schmieders,et al.  Combining SLA prediction and cross layer adaptation for preventing SLA violations , 2011, CloudCom 2011.

[48]  Myra B. Cohen,et al.  Interaction testing of highly-configurable systems in the presence of constraints , 2007, ISSTA '07.

[49]  Javier Tuya,et al.  Testing in Service Oriented Architectures with dynamic binding: A mapping study , 2011, Inf. Softw. Technol..

[50]  Matthias Grochtmann,et al.  Classification trees for partition testing , 1993, Softw. Test. Verification Reliab..

[51]  Tatsuhiro Tsuchiya,et al.  Constructing Test Sets for Pairwise Testing: A SAT-Based Approach , 2011, 2011 Second International Conference on Networking and Computing.

[52]  W. Ziegler,et al.  Extending WS-Agreement with Multi-round Negotiation Capability , 2010 .

[53]  Jacek Czerwonka,et al.  Pairwise Testing in Real World , 2006 .

[54]  Maria Luisa Villani,et al.  Negotiation of Service Level Agreements: An Architecture and a Search-Based Approach , 2007, ICSOC.

[55]  A. Jefferson Offutt,et al.  Combination testing strategies: a survey , 2005, Softw. Test. Verification Reliab..

[56]  Dragan Ivanovic,et al.  Constraint-Based Runtime Prediction of SLA Violations in Service Orchestrations , 2011, ICSOC.

[57]  Manuel Resinas,et al.  Automated Analysis of Conflicts in WS-Agreement , 2014, IEEE Transactions on Services Computing.