Preemptive Regression Testing of Workflow-based Web Services

An external web service may evolve without prior notification. In the course of the regression testing of a workflowbased web service, existing test case prioritization techniques may only verify the latest service composition using the not-yetexecuted test cases, overlooking high-priority test cases that have already been applied to the service composition before the evolution. In this paper, we propose Preemptive Regression Testing (PRT), an adaptive testing approach to addressing this challenge. Whenever a change in the coverage of any service artifact is detected, PRT recursively preempts the current session of regression test and creates a sub-session of the current test session to assure such lately identified changes in coverage by adjusting the execution priority of the test cases in the test suite. Then, the sub-session will resume the execution from the suspended position. PRT terminates only when each test case in the test suite has been executed at least once without any preemption activated in between any test case executions. The experimental result confirms that testing workflow-based web service in the face of such changes is very challenging; and one of the PRT-enriched techniques shows its potential to overcome the challenge.

[1]  Matthias Grochtmann,et al.  Classification trees for partition testing , 1993, Softw. Test. Verification Reliab..

[2]  Tsong Yueh Chen,et al.  On the effectiveness of classification trees for test case construction , 1998, Inf. Softw. Technol..

[3]  Jia Zhang,et al.  An approach to facilitate reliability testing of Web services components , 2004, 15th International Symposium on Software Reliability Engineering.

[4]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[5]  Juan Luo,et al.  Testing Web services by XML perturbation , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[6]  Wei-Tek Tsai,et al.  WSDL-based automatic test case generation for Web services testing , 2005, IEEE International Workshop on Service-Oriented System Engineering (SOSE'05).

[7]  Wei Sun,et al.  BPEL4WS unit testing: framework and implementation , 2005, IEEE International Conference on Web Services (ICWS'05).

[8]  Hong Zhu,et al.  A Framework for Service-Oriented Testing of Web Services , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[9]  Gregg Rothermel,et al.  On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques , 2006, IEEE Transactions on Software Engineering.

[10]  Silvia Regina Vergilio,et al.  Exploring Perturbation Based Testing for Web Services , 2006, 2006 IEEE International Conference on Web Services (ICWS'06).

[11]  Abbas Tarhini,et al.  Regression Testing Web Services-based Applications , 2006, IEEE International Conference on Computer Systems and Applications, 2006..

[12]  Jun Zhu,et al.  Business Process Regression Testing , 2007, ICSOC.

[13]  W. Chan,et al.  A Metamorphic Testing Approach for Online Testing of Service-Oriented Software Applications , 2007, Int. J. Web Serv. Res..

[14]  Yongyan Zheng,et al.  An Automatic Test Case Generation Framework for Web Services , 2007, J. Softw..

[15]  Michael Ruth,et al.  Towards Automatic Regression Test Selection for Web Services , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[16]  T. H. Tse,et al.  Data flow testing of service-oriented workflow applications , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[17]  Eda Marchetti,et al.  Towards Automated WSDL-Based Testing of Web Services , 2008, ICSOC.

[18]  Bixin Li,et al.  Regression Testing of Composite Service: An XBFG-Based Approach , 2008, 2008 IEEE Congress on Services Part II (services-2 2008).

[19]  Tao Xie,et al.  Quota-constrained test-case prioritization for regression testing of service-centric systems , 2008, 2008 IEEE International Conference on Software Maintenance.

[20]  T. H. Tse,et al.  Adaptive Random Test Case Prioritization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[21]  T. H. Tse,et al.  An Empirical Study of the Use of Frankl-Weyuker Data Flow Testing Criteria to Test BPEL Web Services , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[22]  T. H. Tse,et al.  Test case prioritization for regression testing of service-oriented business applications , 2009, WWW '09.

[23]  Baowen Xu,et al.  Test Case Prioritization for Web Service Regression Testing , 2010, 2010 Fifth IEEE International Symposium on Service Oriented System Engineering.

[24]  T. H. Tse,et al.  Taking Advantage of Service Selection: A Study on the Testing of Location-Based Web Services Through Test Case Prioritization , 2010, 2010 IEEE International Conference on Web Services.

[25]  Paolo Tonella,et al.  Test Case Prioritization for Audit Testing of Evolving Web Services Using Information Retrieval Techniques , 2011, 2011 IEEE International Conference on Web Services.

[26]  Eda Marchetti,et al.  Bringing white-box testing to Service Oriented Architectures through a Service Oriented Approach , 2011, J. Syst. Softw..

[27]  T. H. Tse,et al.  XML-manipulating test case prioritization for XML-manipulating services , 2011, J. Syst. Softw..

[28]  Dejan S. Milojicic,et al.  Automatic Determination of Compatibility in Evolving Services , 2011, Int. J. Web Serv. Res..

[29]  Theodore D. Hellmann,et al.  Testing of web services - A systematic mapping , 2012, 2012 IEEE Eighth World Congress on Services.

[30]  Hong Zhu,et al.  Collaborative Testing of Web Services , 2012, IEEE Transactions on Services Computing.

[31]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[32]  Hareton K. N. Leung,et al.  Automatic test case selection for regression testing of composite service based on extensible BPEL flow graph , 2012, J. Syst. Softw..

[33]  Huai Liu,et al.  A Metamorphic Relation-Based Approach to Testing Web Services Without Oracles , 2012, Int. J. Web Serv. Res..

[34]  T. H. Tse,et al.  Preemptive Regression Test Scheduling Strategies: A New Testing Approach to Thriving on the Volatile Service Environments , 2012, 2012 IEEE 36th Annual Computer Software and Applications Conference.

[35]  Bo Jiang,et al.  Prioritizing Structurally Complex Test Pairs for Validating WS-BPEL Evolutions , 2013, 2013 IEEE 20th International Conference on Web Services.

[36]  Bo Jiang,et al.  Test Pair Selection for Test Case Prioritization in Regression Testing for WS-BPEL Programs , 2013, Int. J. Web Serv. Res..

[37]  Javier Tuya,et al.  Multi-dimensional criteria for testing web services transactions , 2013, J. Comput. Syst. Sci..

[38]  Jun Zhu,et al.  Effective Message-Sequence Generation for Testing BPEL Programs , 2013, IEEE Transactions on Services Computing.

[39]  Bo Jiang,et al.  Prioritizing Test Cases for Regression Testing of Location-Based Services: Metrics, Techniques, and Case Study , 2014, IEEE Transactions on Services Computing.

[40]  Fevzi Belli,et al.  A holistic approach to model‐based testing of Web service compositions , 2014, Softw. Pract. Exp..