Input Contract Testing of Graphical User Interfaces

User inputs are critical for the security, safety, and reliability of software systems. This paper proposes a new concept called user input contracts, which is an integral part of a design-by-contract supplemented development process, and a model-based testing approach to detect violations of user input contracts. The approach generates test cases from an input contract integrated with graph-based model of user interface specification and applies them to the system under consideration. The paper presents a proof-of-concept tool that has been developed and used to validate the approach by experiments. The experiments are conducted on a web-based system for marketing tourist services to analyze input robustness of system under consideration with respect to user input contracts.

[1]  Fevzi Belli,et al.  Finite state testing and analysis of graphical user interfaces , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[2]  Pedro Guerreiro Simple support for design by contract in C++ , 2001, Proceedings 39th International Conference and Exhibition on Technology of Object-Oriented Languages and Systems. TOOLS 39.

[3]  Andreas Zeller,et al.  Search-based system testing: high coverage, no false alarms , 2012, ISSTA 2012.

[4]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[5]  R. Kramer iContract - The Java(tm) Design by Contract(tm) Tool , 1998 .

[6]  Ilinca Ciupa,et al.  Strategies for Random Contract-Based Testing , 2008 .

[7]  Robert Gove,et al.  Machine Learning and Event-Based Software Testing: Classifiers for Identifying Infeasible GUI Event Sequences , 2012, Adv. Comput..

[8]  Nina Yevtushenko,et al.  On reducing test length for FSMs with extra states , 2012, Softw. Test. Verification Reliab..

[9]  Stephen H. Edwards,et al.  A framework for detecting interface violations in component-based software , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[10]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[11]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[12]  Fevzi Belli,et al.  Are Longer Test Sequences Always Better? - A Reliability Theoretical Analysis , 2010, 2010 Fourth International Conference on Secure Software Integration and Reliability Improvement Companion.

[13]  Fevzi Belli,et al.  Test minimization for human-computer interaction , 2006, Applied Intelligence.

[14]  A. Jefferson Offutt,et al.  Input validation analysis and testing , 2006, Empirical Software Engineering.

[15]  Fevzi Belli,et al.  Does "Depth" Really Matter? On the Role of Model Refinement for Testing and Reliability , 2011, 2011 IEEE 35th Annual Computer Software and Applications Conference.

[16]  Mario Piattini,et al.  Automated generation of test oracles using a model-driven approach , 2013, Inf. Softw. Technol..

[17]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[18]  Sergiu M. Dascalu,et al.  Challenges and opportunities for improving code-based testing of graphical user interfaces , 2006, J. Comput. Methods Sci. Eng..

[19]  Gregor von Bochmann,et al.  Evaluating Reliability-Testing Usage Models , 2012, 2012 IEEE 36th Annual Computer Software and Applications Conference.

[20]  F. Belli,et al.  On “ Negative ” Tests of Web Applications , 2007 .

[21]  Michael Linschulte,et al.  Cost Reduction through Combining Test Sequences with Input Data , 2012, 2012 IEEE Sixth International Conference on Software Security and Reliability Companion.

[22]  Mary Lou Soffa,et al.  Coverage criteria for GUI testing , 2001, ESEC/FSE-9.

[23]  Dean Wampler,et al.  Contract4J for Design by Contract in Java: Design Pattern-Like Protocols and Aspect Interfaces , 2006 .

[24]  James A. Whittaker Software's Invisible Users , 2001, IEEE Softw..

[25]  Ramana Rao,et al.  Implementational Reflection in Silica , 1991, ECOOP.

[26]  Philippe Collet,et al.  A contracting system for hierarchical components , 2005, CBSE'05.

[27]  José Creissac Campos,et al.  Model-based User Interface Testing With Spec Explorer and ConcurTaskTrees , 2008, Electron. Notes Theor. Comput. Sci..

[28]  Hui Liu,et al.  Covering code behavior on input validation in functional testing , 2009, Inf. Softw. Technol..

[29]  Fevzi Belli,et al.  Event-Based Input Validation Using Design-by-Contract Patterns , 2009, 2009 20th International Symposium on Software Reliability Engineering.

[30]  Richard M. Karp,et al.  Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems , 1972, Combinatorial Optimization.

[31]  Mary Lou Soffa,et al.  Automated test oracles for GUIs , 2000, SIGSOFT '00/FSE-8.

[32]  Jean-Marc Jézéquel,et al.  QCCS: A methodology for the development of contract-aware components based on Aspect Oriented Design , 2003 .

[33]  Len Bass,et al.  Technical Concepts of Component-Based Software Engineering, Volume 2 , 2000 .

[34]  I. Crnkovic,et al.  On the Definition of Concepts in Component Based Software development , 2002 .

[35]  Claude Caci,et al.  Testing object-oriented systems , 2000, SOEN.

[36]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[37]  Ian M. Holland,et al.  Contracts: specifying behavioral compositions in object-oriented systems , 1990, OOPSLA/ECOOP '90.

[38]  Debra J. Richardson,et al.  TAOS: Testing with Analysis and Oracle Support , 1994, ISSTA '94.

[39]  Fevzi Belli,et al.  Event‐based modelling, analysis and testing of user interactions: approach and case study , 2006, Softw. Test. Verification Reliab..

[40]  Ilinca Ciupa,et al.  Automatic Testing Based on Design by Contract TM , 2005 .

[41]  Reinhold Plösch Design by Contract for Python , 1997, APSEC.

[42]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[43]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[44]  Shaoying Liu,et al.  Criteria for generating specification-based tests , 1999, Proceedings Fifth IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'99) (Cat. No.PR00434).

[45]  Weiqun Zheng,et al.  Test by Contract for UML-Based Software Component Testing , 2008, International Symposium on Computer Science and its Applications.

[46]  Reinhold Plösch,et al.  Contracts: from analysis to C++ implementation , 1999, Proceedings of Technology of Object-Oriented Languages and Systems - TOOLS 30 (Cat. No.PR00278).

[47]  Nikolai Tillmann,et al.  A Model-to-Implementation Mapping Tool for Automated Model-Based GUI Testing , 2005, ICFEM.

[48]  Abdul Rauf,et al.  Automated GUI Test Coverage Analysis Using GA , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[49]  Alexandre Petrenko,et al.  Model-based testing of software and systems: recent advances and challenges , 2012, International Journal on Software Tools for Technology Transfer.

[50]  Alfred V. Aho,et al.  An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours , 1991, IEEE Trans. Commun..

[51]  Mao Ye,et al.  Automated Oracle Based on Multi-Weighted Neural Networks for GUI Testing , 2007 .

[52]  Paolo Tonella,et al.  Search-Based Testing of Ajax Web Applications , 2009, 2009 1st International Symposium on Search Based Software Engineering.

[53]  Hong Sun,et al.  Investigating the use of analysis contracts to improve the testability of object‐oriented code , 2003, Softw. Pract. Exp..

[54]  Myra B. Cohen,et al.  Repairing GUI Test Suites Using a Genetic Algorithm , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[55]  Reiko Heckel,et al.  Towards Contract-based Testing of Web Services , 2005, Electron. Notes Theor. Comput. Sci..

[56]  Chengying Mao,et al.  Regression testing for component-based software via built-in test design , 2007, SAC '07.

[57]  A. Jefferson Offutt,et al.  Web application bypass testing , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[58]  Bertrand Meyer,et al.  On the Predictability of Random Tests for Object-Oriented Software , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[59]  Jean-Marc Jézéquel,et al.  Design by Contract to Improve Software Vigilance , 2006, IEEE Transactions on Software Engineering.

[60]  Xuebing Yang,et al.  An FSM based GUI test automation model , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.