UI-design driven model-based testing

Testing interactive systems is a notoriously difficult task. Not only do we need to ensure that the functionality of the developed system is correct with respect to the requirements and specifications, but also we need to ensure that the user interface (UI) to the system is correct (enables a user to access the functionality correctly) and is usable. These different requirements of interactive system testing are not easily combined within a single testing strategy. We investigate the use of models of interactive systems, which have been derived from design artefacts, as the basis for generating tests for an implemented system. We show how automatically generated abstract tests can be used as the basis for a model-based method for testing interactive systems which has low overhead in terms of the models required and which enables testing of UI and system functionality from the perspective of user interaction. We also examine other testing strategies which use the same abstract tests as their basis and discuss general problems in the area of interactive system testing and propose some solutions.

[1]  Judy Bowen,et al.  UI-Design Driven Model-Based Testing , 2009, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[2]  Nikolai Tillmann,et al.  Modeling and Testing Hierarchical GUIs , 2005, Abstract State Machines.

[3]  Harold W. Thimbleby,et al.  User interface model discovery: towards a generic approach , 2010, EICS '10.

[4]  Annika Hinze,et al.  The digital parrot: Combining context-awareness and semantics to augment memory , 2007 .

[5]  Fevzi Belli,et al.  Finite state testing and analysis of graphical user interfaces , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[6]  Fevzi Belli,et al.  A Holistic View for Finite-State Modeling and Testing of User Interactions , 2003 .

[7]  Carl Erickson,et al.  Presenter First: organizing complex GUI applications for test-driven development , 2006, AGILE 2006 (AGILE'06).

[8]  Judy Bowen,et al.  Formal Models for Informal GUI Designs , 2007, FMIS@ICFEM.

[9]  Atif M. Memon,et al.  Model-Based Testing of Community-Driven Open-Source GUI Applications , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[10]  BowenJudy,et al.  Formal Models for Informal GUI Designs , 2007 .

[11]  Alex Ruiz,et al.  Test-Driven GUI Development with TestNG and Abbot , 2007, IEEE Software.

[12]  Myra B. Cohen,et al.  Towards Dynamic Adaptive Automated Test Generation for Graphical User Interfaces , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[13]  Bruno Legeard,et al.  Controlling test case explosion in test generation from B formal models: Research Articles , 2004 .

[14]  Judy Bowen,et al.  UI-driven test-first development of interactive systems , 2011, EICS '11.

[15]  Lee J. White,et al.  Generating test cases for GUI responsibilities using complete interaction sequences , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[16]  Robert M. Hierons.  Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[17]  Ana C. R. Paiva,et al.  Towards the Integration of Visual and Formal Models for GUI Testing , 2007, MBT.

[18]  Jüri Vain,et al.  Model-Based Testing of Reactive Systems , 2009 .

[19]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[20]  João Saraiva,et al.  The GUISurfer tool: towards a language independent approach to reverse engineering GUI code , 2010, EICS '10.

[21]  Alexander Pretschner,et al.  Methodological Issues in Model-Based Testing , 2004, Model-Based Testing of Reactive Systems.

[22]  Judy Bowen,et al.  Formal models for user interface design artefacts , 2008, Innovations in Systems and Software Engineering.

[23]  Kent L. Beck,et al.  Aim, Fire , 2001, IEEE Softw..

[24]  Judy Bowen,et al.  A Tale of Two Studies , 2013, AUIC.

[25]  Atif M. Memon,et al.  An event‐flow model of GUI‐based applications for testing , 2007, Softw. Test. Verification Reliab..

[26]  Atif M. Memon,et al.  Using Reverse Engineering for Automated Usability Evaluation of Gui-Based Applications , 2009, Human-Centered Software Engineering.

[27]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[28]  Judy Bowen,et al.  Refinement for User Interface Designs , 2008, Electron. Notes Theor. Comput. Sci..

[29]  Judy Bowen,et al.  Formal Models and Refinement for Graphical User Interface Design , 2008 .

[30]  David A. Carrington,et al.  A Framework for Specification-Based Testing , 1996, IEEE Trans. Software Eng..

[31]  Judy Bowen,et al.  Developing usability studies via formal models of UIs , 2010, EICS '10.

[32]  Bruno Legeard,et al.  Controlling test case explosion in test generation from B formal models , 2004, Softw. Test. Verification Reliab..

[33]  Judy Bowen,et al.  Modelling user manuals of modal medical devices and learning from the experience , 2012, EICS '12.