Event-driven web application testing based on model-based mutation testing

Abstract Context Event-Driven Software (EDS) is a class of software whose behavior is driven by incoming events. Web and desktop applications that respond to user-initiated events on their Graphical User Interface (GUI), or embedded software responding to events and signals received from the equipment in its operating environment are examples of EDS. Testing EDS poses great challenges to software testers. One of these challenges is the need to generate a huge number of possible event sequences that could sufficiently cover the EDS’s state space. Objective In this paper, we introduce a new six-stage testing procedure for event-driven web applications to overcome EDS testing challenges. Method The stages of the testing procedure include dividing the application based on its structure, creating functional graphs for each section, creating mutants from functional graphs, choosing coverage criteria to produce test paths, merging event sequences to make longer ones, and deriving and running test cases. We have analyzed our proposed testing procedure with the help of four metrics consisting of Fault Detection Density (FDD), Fault Detection Effectiveness (FDE), Mutation Score, and Unique Fault. Results Using this procedure, we have prepared prioritized test cases and also discovered a list of unique faults by running the suggested test cases on a sample real-world web application called Academic E-mail System. Conclusion We propose that our suggested testing procedure has some advantages such as creating functional graphs with requirements document, resolving the problem of removing infeasible test cases with these graphs and conditions on the “add edge” operator before creating mutants. But the suggested testing procedure, like any other method, had some drawbacks. Because most of the stages in the approach were performed manually, the testing time was increased.

[1]  Fevzi Belli,et al.  Event-Based Mutation Testing vs. State-Based Mutation Testing - An Experimental Comparison , 2011, 2011 IEEE 35th Annual Computer Software and Applications Conference.

[2]  Paolo Tonella,et al.  Statistical testing of Web applications , 2004, J. Softw. Maintenance Res. Pract..

[3]  Myra B. Cohen,et al.  Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces , 2006, IEEE Trans. Software Eng..

[4]  Linda Dailey Paulson,et al.  Building Rich Web Applications with Ajax , 2005, Computer.

[5]  Fevzi Belli,et al.  A Formal Framework for Mutation Testing , 2010, 2010 Fourth International Conference on Secure Software Integration and Reliability Improvement.

[6]  Myra B. Cohen,et al.  Towards Dynamic Adaptive Automated Test Generation for Graphical User Interfaces , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[7]  A. J. Offutt A practical system for mutation testing: help for the common programmer , 1994, Proceedings., International Test Conference.

[8]  G.S. Nezlek,et al.  Rich Internet Applications The Next Stage of Application Development , 2007, 2007 29th International Conference on Information Technology Interfaces.

[9]  Florian Lorber,et al.  Time for Mutants - Model-Based Mutation Testing with Timed Automata , 2013, TAP@STAF.

[10]  Roy P. Pargas,et al.  Test‐data generation using genetic algorithms , 1999 .

[11]  A. Isabella,et al.  Study Paper on Test Case generation for GUI Based Testing , 2012, ArXiv.

[12]  Fevzi Belli,et al.  Event‐based modelling, analysis and testing of user interactions: approach and case study , 2006, Softw. Test. Verification Reliab..

[13]  Atif M. Memon,et al.  Developing a Single Model and Test Prioritization Strategies for Event-Driven Software , 2011, IEEE Transactions on Software Engineering.

[14]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[15]  Atif M. Memon,et al.  Using GUI Run-Time State as Feedback to Generate Test Cases , 2007, 29th International Conference on Software Engineering (ICSE'07).

[16]  James Miller,et al.  A practical approach to testing GUI systems , 2006, Empirical Software Engineering.

[17]  Jens Grabowski,et al.  A Model for Usage-Based Testing of Event-Driven Software , 2011, 2011 Fifth International Conference on Secure Software Integration and Reliability Improvement - Companion.

[18]  Myra B. Cohen,et al.  GUI Interaction Testing: Incorporating Event Context , 2011, IEEE Transactions on Software Engineering.

[19]  L.D. Paulson Will hard drives finally stop shrinking? , 2005, Computer.

[20]  Atif M. Memon,et al.  Generating Event Sequence-Based Test Cases Using GUI Runtime State Feedback , 2010, IEEE Transactions on Software Engineering.

[21]  Myra B. Cohen,et al.  Repairing GUI Test Suites Using a Genetic Algorithm , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[22]  Patrick Harms,et al.  AutoQUEST -- Automated Quality Engineering of Event-Driven Software , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[23]  Kent D. Lee Python Programming Fundamentals , 2014, Undergraduate Topics in Computer Science.

[24]  Myra B. Cohen,et al.  Covering array sampling of input event sequences for automated gui testing , 2007, ASE.

[25]  Fevzi Belli,et al.  Communication Sequence Graphs for Mutation-Oriented Integration Testing , 2009, 2009 Third IEEE International Conference on Secure Software Integration and Reliability Improvement.

[26]  Mary Lou Soffa,et al.  Hierarchical GUI Test Case Generation Using Automated Planning , 2001, IEEE Trans. Software Eng..

[27]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[28]  Giovanni Toffetti Carughi,et al.  Conceptual modeling and code generation for rich internet applications , 2006, ICWE '06.

[29]  Jirapun Daengdej,et al.  A Test Case Prioritization Method with Practical Weight Factors , 2010 .

[30]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[31]  J. R. Okin The Internet Revolution: The Not-for-Dummies Guide to the History, Technology, and Use of the Internet , 2005 .

[32]  Atif M. Memon,et al.  Using a pilot study to derive a GUI model for automated testing , 2008, TSEM.