An Experimental Evaluation of Web Mutation Operators

While modern web development technologies enhance the capabilities of web applications, they introduce challenges for testers. This paper introduces, evaluates, and refines web mutation operators that target interaction faults in web applications. An experimental study is conducted on 11 subject web applications using 15 web mutation operators. The effectiveness of 12 independently developed test sets are analyzed in terms of how well they kill web mutants. Web mutation adequate tests are compared with independently created test sets to evaluate the web mutation operators. Tests designed to satisfy the web mutation testing criterion provide 100% coverage while the tests designed to satisfy traditional testing criteria provide, on average, 47%coverage. The paper also analyzes which mutants and mutation operators the traditional tests had difficulty killing. We found that some types of mutants that are not particularly hard to kill were missed by all traditional tests. Additionally, web mutation testing produces very few equivalent mutants.

[1]  A. Jefferson Offutt,et al.  Applying Mutation Testing to Web Applications , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[2]  Massimiliano Di Penta,et al.  Considering browser interaction in Web application testing , 2003, Fifth IEEE International Workshop on Web Site Evolution, 2003. Theme: Architecture. Proceedings..

[3]  Ali Mesbah,et al.  Efficient JavaScript Mutation Testing , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[4]  Alessandro Orso,et al.  Precise interface identification to improve testing and analysis of web applications , 2009, ISSTA.

[5]  Paolo Tonella,et al.  Analysis and testing of Web applications , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[6]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[7]  Yves Le Traon,et al.  Tailored Shielding and Bypass Testing of Web Applications , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[8]  Shinichi Honiden,et al.  Mutation analysis for Java script web application testing , 2013, ICSE 2013.

[9]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[10]  Paolo Tonella,et al.  Statistical testing of Web applications , 2004, J. Softw. Maintenance Res. Pract..

[11]  James Miller,et al.  Agile security testing of Web-based systems via HTTPUnit , 2005, Agile Development Conference (ADC'05).

[12]  Paolo Tonella,et al.  Empirical Validation of a Web Fault Taxonomy and its usage for Fault Seeding , 2007, 2007 9th IEEE International Workshop on Web Site Evolution.

[13]  A. Jefferson Offutt,et al.  Testing Web applications by modeling with FSMs , 2005, Software & Systems Modeling.

[14]  A. Jefferson Offutt,et al.  Bypass testing of Web applications , 2004, 15th International Symposium on Software Reliability Engineering.

[15]  Shinichi Honiden,et al.  Mutation Analysis for JavaScriptWeb Application Testing , 2013, SEKE.

[16]  Sreedevi Sampath,et al.  Web application fault classification - an exploratory study , 2008, ESEM '08.

[17]  A. Jefferson Offutt,et al.  Quality Attributes of Web Software Applications , 2002, IEEE Softw..

[18]  A. Jefferson Offutt,et al.  Modeling presentation layers of web applications for testing , 2009, Software & Systems Modeling.

[19]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[20]  Lance A. Liotta,et al.  A wireless Internet application for healthcare , 2001, Proceedings 14th IEEE Symposium on Computer-Based Medical Systems. CBMS 2001.

[21]  A. Jefferson Offutt,et al.  Establishing Theoretical Minimal Sets of Mutants , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[22]  David Chenho Kung,et al.  Structural testing of Web applications , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[23]  A. Jefferson Offutt,et al.  Mutation 2000: uniting the orthogonal , 2001 .