Optimised Realistic Test Input Generation Using Web Services

We introduce a multi-objective formulation of service-oriented testing, focusing on the balance between service price and reliability. We experimented with NSGA-II for this problem, investigating the effect on performance and quality of composition size, topology and the number of services discovered. For topologies small enough for exhaustive search we found that NSGA-II finds a pareto front very near (the fronts are a Euclidean distance of ~0.00024 price-reliability points apart) the true pareto front. Regarding performance, we find that composition size has the strongest effect, with smaller topologies consuming more machine time; a curious effect we believe is due to the influence of crowding distance. Regarding result quality, our results reveal that size and topology have more effect on the front found than the number of service choices discovered. As expected the price-reliability relationship (logarithmic, linear, exponential) is replicated in the front discovered when correlation is high, but as the price-reliability correlation decreases, we find fewer solutions on the front and the front becomes less smooth. © 2012 Springer-Verlag.

[1]  Mark Harman,et al.  Automated web application testing using search based software engineering , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[2]  Mark Harman,et al.  Automatically generating realistic test input from web services , 2011, Proceedings of 2011 IEEE 6th International Symposium on Service Oriented System (SOSE).

[3]  Norbert Oster,et al.  Automatic Test Data Generation by Multi-objective Optimisation , 2006, SAFECOMP.

[4]  Mark Harman,et al.  The Current State and Future of Search Based Software Engineering , 2007, Future of Software Engineering (FOSE '07).

[5]  Mark Grechanik,et al.  Automatic Test Generation From GUI Applications For Testing Web Services , 2007, 2007 IEEE International Conference on Software Maintenance.

[6]  Yuanyuan Zhang,et al.  The multi-objective next release problem , 2007, GECCO '07.

[7]  Mark Harman,et al.  Optimised Realistic Test Input Generation , 2011 .

[8]  Phil McMinn,et al.  Search-Based Test Input Generation for String Data Types Using the Results of Web Queries , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[9]  Gregg Rothermel,et al.  Leveraging user-session data to support Web application testing , 2005, IEEE Transactions on Software Engineering.

[10]  Mark Harman,et al.  A multi-objective approach to search-based test data generation , 2007, GECCO '07.

[11]  Raquel Blanco,et al.  A First Approach to Test Case Generation for BPEL Compositions of Web Services Using Scatter Search , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[12]  Gerardo Canfora,et al.  Search-based testing of service level agreements , 2007, GECCO '07.

[13]  Xin Yao,et al.  Handling Constraints for Search Based Software Test Data Generation , 2008, 2008 IEEE International Conference on Software Testing Verification and Validation Workshop.

[14]  Mei-Hwa Chen,et al.  Clustering and Tailoring User Session Data for Testing Web Applications , 2009, 2009 International Conference on Software Testing Verification and Validation.

[15]  Mark Harman,et al.  Automated Session Data Repair for Web Application Regression Testing , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[16]  Silvia Regina Vergilio,et al.  A Multi-Objective Genetic Algorithm to Test Data Generation , 2010, 2010 22nd IEEE International Conference on Tools with Artificial Intelligence.

[17]  Mark Harman,et al.  Testing and verification in service‐oriented architecture: a survey , 2013, Softw. Test. Verification Reliab..

[18]  Mark Harman,et al.  Reducing qualitative human oracle costs associated with automatically generated test data , 2010, STOV '10.

[19]  Mark Harman,et al.  Pareto efficient multi-objective test case selection , 2007, ISSTA '07.

[20]  Wan Nurhayati Wan Ab. Rahman,et al.  Challenges to describe QoS requirements for web services quality prediction to support web services interoperability in electronic commerce , 2008 .