Whitening SOA Testing via Event Exposure

Whitening the testing of service-oriented applications can provide service consumers confidence on how well an application has been tested. However, to protect business interests of service providers and to prevent information leakage, the implementation details of services are usually invisible to service consumers. This makes it challenging to determine the test coverage of a service composition as a whole and design test cases effectively. To address this problem, we propose an approach to whiten the testing of service compositions based on events exposed by services. By deriving event interfaces to explore only necessary test coverage information from service implementations, our approach allows service consumers to determine test coverage based on selected events exposed by services at runtime without releasing the service implementation details. We also develop an approach to design test cases effectively based on event interfaces concerning both effectiveness and information leakage. The experimental results show that our approach outperforms existing testing approaches for service compositions with up to 49 percent more test coverage and an up to 24 percent higher fault-detection rate. Moreover, our solution can trade off effectiveness, efficiency, and information leakage for test case generation.

[1]  Jun Wei,et al.  Detecting Data Inconsistency Failure of Composite Web Services Through Parametric Stateful Aspect , 2010, 2010 IEEE International Conference on Web Services.

[2]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[3]  Eda Marchetti,et al.  Bringing white-box testing to Service Oriented Architectures through a Service Oriented Approach , 2011, J. Syst. Softw..

[4]  E. Harder,et al.  Apache , 1965 .

[5]  Eda Marchetti,et al.  Whitening SOA testing , 2009, ESEC/FSE '09.

[6]  Carlo Ghezzi,et al.  Change-point detection for black-box services , 2010, FSE '10.

[7]  Leonardo Mariani Behavior capture and test for verifying evolving component-based systems , 2004, Proceedings. 26th International Conference on Software Engineering.

[8]  Alexander L. Wolf,et al.  Evaluating Test Suites and Adequacy Criteria Using Simulation-Based Models of Distributed Systems , 2008, IEEE Transactions on Software Engineering.

[9]  Elisabeth Oswald,et al.  Pinpointing side-channel information leaks in web applications , 2012, Journal of Cryptographic Engineering.

[10]  Tom Chothia,et al.  Statistical Measurement of Information Leakage , 2010, TACAS.

[11]  Rocco De Nicola,et al.  Testing Equivalence for Processes , 1983, ICALP.

[12]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[13]  Liliana Ardissono,et al.  Enhancing Web services with diagnostic capabilities , 2005, Third European Conference on Web Services (ECOWS'05).

[14]  Jun Wei,et al.  Runtime Monitoring of Data-centric Temporal Properties for Web Services , 2011, 2011 IEEE International Conference on Web Services.

[15]  Henry Muccini,et al.  Using software architecture for code testing , 2004, IEEE Transactions on Software Engineering.

[16]  Johannes Mayer,et al.  An empirical analysis and comparison of random testing techniques , 2006, ISESE '06.

[17]  Lionel C. Briand,et al.  A comprehensive and systematic methodology for client-server class integration testing , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[18]  Hans-Arno Jacobsen,et al.  A distributed service-oriented architecture for business process execution , 2010, TWEB.

[19]  Lionel C. Briand,et al.  Automated, contract-based user testing of commercial-off-the-shelf components , 2006, ICSE '06.

[20]  Hans-Arno Jacobsen,et al.  White-Box Testing of Service Compositions Via Event Interfaces , 2010 .

[21]  David Luckham,et al.  The power of events - an introduction to complex event processing in distributed enterprise systems , 2002, RuleML.

[22]  Jun Zhu,et al.  Business-process-driven gray-box SOA testing , 2008, IBM Syst. J..

[23]  C. Peltz,et al.  Web Services Orchestration and Choreography , 2003, Computer.

[24]  Thomas A. Henzinger,et al.  Web service interfaces , 2005, WWW '05.

[25]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[26]  Wei Sun,et al.  A Graph-Search Based Approach to BPEL4WS Test Generation , 2006, 2006 International Conference on Software Engineering Advances (ICSEA'06).

[27]  W. Chan,et al.  A Metamorphic Testing Approach for Online Testing of Service-Oriented Software Applications , 2007, Int. J. Web Serv. Res..

[28]  Walid Gaaloul,et al.  Event-Based Design and Runtime Verification of Composite Service Transactional Behavior , 2010, IEEE Transactions on Services Computing.

[29]  David L. Dill,et al.  A Decision Procedure for Bit-Vectors and Arrays , 2007, CAV.

[30]  Phil McMinn,et al.  Search-based software test data generation: a survey: Research Articles , 2004 .

[31]  Xiaoying Bai,et al.  Contract-Based Testing for Web Services , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[32]  Corina S. Pasareanu,et al.  Assume-Guarantee Verification for Interface Automata , 2008, FM.

[33]  Dimitris Dranidis,et al.  Run-time Verification of Behavioural Conformance for Conversational Web Services , 2009, 2009 Seventh IEEE European Conference on Web Services.

[34]  Gerardo Canfora,et al.  Testing services and service-centric systems: challenges and opportunities , 2006, IT Professional.

[35]  T. H. Tse,et al.  Data flow testing of service choreography , 2009, ESEC/FSE '09.

[36]  Fatiha Zaïdi,et al.  A Formal Framework for Service Orchestration Testing Based on Symbolic Transition Systems , 2009, TestCom/FATES.

[37]  Shing-Chi Cheung,et al.  Publishing and composition of atomicity-equivalent services for B2B collaboration , 2006, ICSE.

[38]  Hong Zhu,et al.  Collaborative Testing of Web Services , 2012, IEEE Transactions on Services Computing.

[39]  Elaine J. Weyuker,et al.  Axiomatizing software test data adequacy , 1986, IEEE Transactions on Software Engineering.

[40]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs, and Quiescence , 1996, TACAS.

[41]  Alessandro Orso,et al.  Component Metadata for Software Engineering Tasks , 2000, EDO.

[42]  Wei-Tek Tsai,et al.  WSDL-based automatic test case generation for Web services testing , 2005, IEEE International Workshop on Service-Oriented System Engineering (SOSE'05).

[43]  Volker Gruhn,et al.  An integrated testing technique for component-based software , 2001, Proceedings ACS/IEEE International Conference on Computer Systems and Applications.

[44]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[45]  T. H. Tse,et al.  Data flow testing of service-oriented workflow applications , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[46]  Yongyan Zheng,et al.  An Automatic Test Case Generation Framework for Web Services , 2007, J. Softw..

[47]  Kathrin Kaschner,et al.  Automatic Test Case Generation for Interacting Services , 2009, ICSOC Workshops.

[48]  Francesca Lonetti,et al.  Approaches to Functional, Structural and Security SOA Testing , 2012 .

[49]  Shing-Chi Cheung,et al.  Atomicity Analysis of Service Composition across Organizations , 2009, IEEE Transactions on Software Engineering.

[50]  T. H. Tse,et al.  Tag-Based Techniques for Black-Box Test Case Prioritization for Service Testing , 2009, 2009 Ninth International Conference on Quality Software.

[51]  Juan Luo,et al.  Testing Web services by XML perturbation , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[52]  K. Krippendorff Mathematical Theory of Communication , 2009 .

[53]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[54]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[55]  Hernán C. Melgratti,et al.  Testing Service Composition , 2007 .

[56]  Tevfik Bultan,et al.  Runtime Verification of Web Service Interface Contracts , 2010, Computer.

[57]  Santhosh Kumaran,et al.  Conversation support for business process integration , 2002, Proceedings. Sixth International Enterprise Distributed Object Computing.

[58]  Eda Marchetti,et al.  Towards Automated WSDL-Based Testing of Web Services , 2008, ICSOC.

[59]  Lori A. Clarke,et al.  A System to Generate Test Data and Symbolically Execute Programs , 1976, IEEE Transactions on Software Engineering.

[60]  Juan-Zi Li,et al.  Ontology-Based Test Case Generation for Testing Web Services , 2007, Eighth International Symposium on Autonomous Decentralized Systems (ISADS'07).

[61]  J. L. Hodges,et al.  Estimates of Location Based on Rank Tests , 1963 .

[62]  James C. Corbett,et al.  Bandera: extracting finite-state models from Java source code , 2000, ICSE.

[63]  Andrea Polini,et al.  SOA Test Governance: Enabling Service Integration Testing across Organization and Technology Borders , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[64]  Taratip Suwannasart,et al.  Web Service Test Case Generation Based on Decision Table (Short Paper) , 2008, 2008 The Eighth International Conference on Quality Software.

[65]  T. H. Tse,et al.  Test case prioritization for regression testing of service-oriented business applications , 2009, WWW '09.

[66]  Wei Sun,et al.  BPEL4WS unit testing: framework and implementation , 2005, IEEE International Conference on Web Services (ICWS'05).

[67]  Leonardo Mariani,et al.  Compatibility and Regression Testing of COTS-Component-Based Software , 2007, 29th International Conference on Software Engineering (ICSE'07).

[68]  Wu Chou,et al.  Control Flow Analysis and Coverage Driven Testing for Web Services , 2008, 2008 IEEE International Conference on Web Services.

[69]  Berndt Farwer,et al.  ω-automata , 2002 .

[70]  Tao Zhang,et al.  WSDL-Based Automated Test Data Generation for Web Service , 2008, 2008 International Conference on Computer Science and Software Engineering.

[71]  Tien-Dung Cao,et al.  Automated Runtime Verification for Web Services , 2010, 2010 IEEE International Conference on Web Services.

[72]  Gerardo Canfora,et al.  Service-Oriented Architectures Testing: A Survey , 2009, ISSSE.

[73]  Sebastián Uchitel,et al.  Model-based verification of Web service compositions , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[74]  Manoj Kumar Sahu,et al.  TEST CASE PRIORITIZATION FOR REGRESSION TESTING , 2014 .

[75]  I. Melzer Web Services Description Language , 2010 .

[76]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[77]  Xiang Fu,et al.  Analysis of interacting BPEL web services , 2004, WWW '04.

[78]  Scott D. Stoller,et al.  Runtime analysis for atomicity for multi-threaded programs , 2005 .

[79]  A. Jefferson Offutt,et al.  Coupling-based Criteria for Integration Testing , 1998, Softw. Test. Verification Reliab..

[80]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[81]  Andrea Polini,et al.  Model-Based Generation of Testbeds for Web Services , 2008, TestCom/FATES.