A Comparative Study of Software Testing Techniques

Nowadays, software systems have become an essential element in our daily life. To ensure the quality and operation of software, testing activities have become primordial in the software development life cycle (SDLC). Indeed, software bugs can potentially cause dramatic consequences if the product is released to the end user without testing. The software testing role is to verify that the actual result and the expected result are consistent and ensure that the system is delivered without bugs. Many techniques, approaches and tools have been proposed to help check that the system is defect free. In this paper, we highlight two software testing techniques considered among the most used techniques to perform software tests, and then we perform a comparative study of these techniques, the approaches that supports studied techniques, and the tools used for each technique. We have selected the first technique based on the 2014 survey [62] that heighted the motivations for using the Model-based-testing, and by analyzing the survey results we have found that some MBT limits are benefits in Risk based testing, the second technique in our study.

[1]  Xiaoying Bai,et al.  Risk-Based Adaptive Group Testing of Semantic Web Services , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[2]  Rajesh Subramanyan,et al.  A survey on model-based testing approaches: a systematic review , 2007, WEASELTech '07.

[3]  Neil Thompson,et al.  Risk Based E-Business Testing , 2002 .

[4]  Hendrik Brinksma,et al.  Côte de Resyste : Automated Model Based Testing , 2002 .

[5]  Thomas Bauer,et al.  Risk-based Statistical Testing: A Refinement- based Approach to the Reliability Analysis of Safety-Critical Systems , 2009 .

[6]  Ahmad Hosseingholizadeh.  A source-based risk analysis approach for software test optimization , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[7]  Dianxiang Xu,et al.  Automated Security Test Generation with Formal Threat Models , 2012, IEEE Transactions on Dependable and Secure Computing.

[8]  Ruth Breu,et al.  Towards Risk -- Driven Security Testing of Service Centric Systems , 2012, 2012 12th International Conference on Quality Software.

[9]  Alan Hartman,et al.  The AGEDIS tools for model based testing , 2004, ISSTA '04.

[10]  Felix Redmill.  Exploring risk-based testing and its implications: Research Articles , 2004 .

[11]  Marc-Florian Wendland,et al.  A Systematic Approach to Risk-Based Testing Using Risk-annotated Requirements Models , 2012, ICSEA 2012.

[12]  Florentin Ipate,et al.  JSXM: A Tool for Automated Test Generation , 2012, SEFM.

[13]  Felix Redmill,et al.  Exploring risk‐based testing and its implications , 2004, Softw. Test. Verification Reliab..

[14]  K. Murthy,et al.  Leveraging Risk Based Testing in Enterprise Systems Security Validation , 2009, 2009 First International Conference on Emerging Network Intelligence.

[15]  Thomas Bauer,et al.  Automated Risk-Based Testing by Integrating Safety Analysis Information into System Behavior Models , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops.

[16]  Siddhartha R. Dalal,et al.  Model-based testing in practice , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[17]  Alexander Pretschner,et al.  One evaluation of model-based testing and its automation , 2005, ICSE.

[18]  Myra B. Cohen,et al.  An orchestrated survey of methodologies for automated software test case generation , 2013, J. Syst. Softw..

[19]  Robert Eschbach,et al.  Risk-Based Testing of Safety-Critical Embedded Systems Driven by Fault Tree Analysis , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[20]  Byoungju Choi,et al.  A Test Case Prioritization Based on Degree of Risk Exposure and its Empirical Study , 2011, Int. J. Softw. Eng. Knowl. Eng..

[21]  Xiaoying Bai,et al.  Risk Assessment and Adaptive Group Testing of Semantic Web Services , 2012, Int. J. Softw. Eng. Knowl. Eng..

[22]  Linda H. Rosenberg,et al.  Risk-Based Object Oriented Testing , 2000 .

[23]  Bernhard K. Aichernig,et al.  MoMut::UML Model-Based Mutation Testing for UML , 2015, 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST).

[24]  Nikhil Kumar,et al.  Enabling the adoption of aspects - testing aspects: a risk model, fault model and patterns , 2009, AOSD '09.

[25]  Felix Redmill.  Theory and practice of risk‐based testing , 2005, Softw. Test. Verification Reliab..

[26]  Michael Felderer,et al.  Experiences and Challenges of Introducing Risk-Based Testing in an Industrial Project , 2013, SWQD.

[27]  Asif Irshad Khan,et al.  Risk-based Testing Techniques: A Perspective Study , 2013 .

[28]  Andreas Ulrich,et al.  Model-based testing in industry: a case study with two MBT tools , 2010, AST '10.

[29]  Alexander Pretschner.  Model-based testing , 2005, ICSE '05.

[30]  Lars Kristoffer Ulstein Jørgensen.  A software tool for risk-based testing , 2005 .

[31]  Fabrice Bouquet,et al.  Mastering Test Generation from Smart Card Software Formal Models , 2004, CASSIS.

[32]  Ruth Breu,et al.  Integrating Manual and Automatic Risk Assessment for Risk-Based Testing , 2012, SWQD.

[33]  Mario Gleirscher,et al.  Hazard Analysis for Technical Systems , 2013, SWQD.

[34]  Yu Qi,et al.  Source code-based software risk assessing , 2005, SAC '05.

[35]  Klaus Pohl,et al.  An automated technique for risk-based test case generation and prioritization , 2008, AST '08.

[36]  Ståle Amland.  Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study , 2000, J. Syst. Softw..

[37]  Kanmani Selvadurai,et al.  Risk-driven security testing using risk analysis with threat modeling approach , 2014, SpringerPlus.

[38]  Yanping Chen,et al.  Specification-based regression test selection with risk analysis , 2002, CASCON.

[39]  Yan Li,et al.  Approaches for the combined use of risk analysis and testing: a systematic literature review , 2014, International Journal on Software Tools for Technology Transfer.

[40]  Bo Zhang,et al.  Introducing model-based testing in an industrial scrum project , 2012, 2012 7th International Workshop on Automation of Software Test (AST).

[41]  Cristine Martins Gomes de Gusmão,et al.  Risk-Based Testing: A Case Study , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[42]  Javier Tuya,et al.  Testing Long-Lived Web Services Transactions Using a Risk-Based Approach , 2010, 2010 10th International Conference on Quality Software.

[43]  Durga Prasad Mohapatra,et al.  Risk analysis: a guiding force in the improvement of testing , 2013, IET Softw..

[44]  Mario Gleirscher,et al.  Hazard-based selection of test cases , 2011, AST '11.

[45]  Bruno Legeard,et al.  A taxonomy of model-based testing , 2006 .

[46]  Norman F. Schneidewind.  RISK-DRIVEN SOFTWARE TESTING AND RELIABILITY , 2007 .

[47]  Ruth Breu,et al.  A Classification for Model-Based Security Testing , 2011 .

[48]  Philipp Zech.  Risk-Based Security Testing in Cloud Computing Environments , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[49]  Ina Schieferdecker,et al.  A taxonomy of risk-based testing , 2014, International Journal on Software Tools for Technology Transfer.

[50]  Javier Tuya,et al.  A Framework to Test Advanced Web Services Transactions , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.