JSART: JavaScript Assertion-Based Regression Testing

Web 2.0 applications rely heavily on JavaScript and client-side runtime manipulation of the DOM tree. One way to provide assurance about the correctness of such highly evolving and dynamic applications is through regression testing. However, JavaScript is loosely typed, dynamic, and notoriously challenging to analyze and test. We propose an automated technique for JavaScript regression testing, which is based on on-the-fly JavaScript source code instrumentation and dynamic analysis to infer invariant assertions. These obtained assertions are injected back into the JavaScript code to uncover regression faults in subsequent revisions of the web application under test. Our approach is implemented in a tool called Jsart. We present our case study conducted on nine open source web applications to evaluate the proposed approach. The results show that our approach is able to effectively generate stable assertions and detect JavaScript regression faults with a high degree of accuracy and minimal performance overhead.

[1]  Arie van Deursen,et al.  Crawling Ajax-Based Web Applications through Dynamic Analysis of User Interface State Changes , 2012, TWEB.

[2]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[3]  Baowen Xu,et al.  Regression testing for Web applications based on slicing , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[4]  MeyerBertrand,et al.  Design by Contract , 1997 .

[5]  Ali Mesbah,et al.  Automatic invariant detection in dynamic web applications , 2010 .

[6]  Paolo Tonella,et al.  State-Based Testing of Ajax Web Applications , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[7]  Ali Mesbah,et al.  AutoFLox: An Automatic Fault Localizer for Client-Side JavaScript , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[8]  John A. Clark,et al.  Searching for invariants using genetic programming and mutation testing , 2011, GECCO '11.

[9]  Xiangyu Zhang,et al.  Statically locating web application bugs caused by asynchronous calls , 2011, WWW.

[10]  Karthik Pattabiraman,et al.  DoDOM: Leveraging DOM Invariants for Web 2.0 Application Robustness Testing , 2010, 2010 IEEE 21st International Symposium on Software Reliability Engineering.

[11]  Emily Hill,et al.  Automated replay and failure detection for web applications , 2005, ASE '05.

[12]  Zhiming Liu,et al.  Theoretical Aspects of Computing - ICTAC 2004, First International Colloquium, Guiyang, China, September 20-24, 2004, Revised Selected Papers , 2005, ICTAC.

[13]  Arie van Deursen,et al.  Automated security testing of web widget interactions , 2009, ESEC/FSE '09.

[14]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[15]  Marat Boshernitsan,et al.  From daikon to agitator: lessons and challenges in building a commercial tool for developer testing , 2006, ISSTA '06.

[16]  Nikolai Tillmann,et al.  DySy: dynamic symbolic execution for invariant inference , 2008, ICSE.

[17]  Frank Tip,et al.  A framework for automated testing of javascript web applications , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[18]  Arie van Deursen,et al.  Invariant-Based Automatic Testing of Modern Web Applications , 2012, IEEE Transactions on Software Engineering.

[19]  David S. Rosenblum,et al.  A historical perspective on runtime assertion checking in software development , 2006, SOEN.

[20]  Mark Harman,et al.  Automated Session Data Repair for Web Application Regression Testing , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[21]  Enric Rodríguez-Carbonell,et al.  Program Verification Using Automatic Generation of Invariants , 2004, ICTAC.

[22]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[23]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[24]  Arie van Deursen,et al.  Regression Testing Ajax Applications: Coping with Dynamism , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[25]  A. Tarhini,et al.  Regression Testing Web Applications , 2008, 2008 International Conference on Advanced Computer Theory and Engineering.

[26]  Shriram Krishnamurthi,et al.  Using static analysis for Ajax intrusion detection , 2009, WWW '09.

[27]  Benjamin Livshits,et al.  GATEKEEPER: Mostly Static Enforcement of Security and Reliability Policies for JavaScript Code , 2009, USENIX Security Symposium.

[28]  Gregg Rothermel,et al.  Leveraging user-session data to support Web application testing , 2005, IEEE Transactions on Software Engineering.

[29]  Sudheendra Hangal,et al.  Tracking down software bugs using automatic anomaly detection , 2002, ICSE '02.