Apoio à automatização de oráculos de teste para programas com interfaces gráficas

Strategies for automated software testing activities are well accepted by both industry and the academy. Essential elements for automation of testing are test oracles. Oracles, which may be mechanisms, functions, parallel executions, etc., are crucial in determining whether the output of an application under test is correct. The automation of oracles is critical when the output system manifested by non-trivial formats, for example, a Graphical User Interface (GUI). For these cases, traditional testing strategies tend to be costly and require considerable efforts of the testers. This master thesis proposes an alternative method for the automation of test oracles for systems with GUIs. To this end, we explore the concepts of Content-Based Image Retrieval for the composition of an automated method called Graphical Oracles (Gr-O). As a contribution, we developed characteristics extractors of GUIs. The conduct and analysis of empirical studies have shown that using of Gr-O can reduce costs for definitions of test oracles for systems with GUIs. Thus, the proposed method may be alternative or complementary to traditional testing techniques found in the literature.

[1]  Nikolas P. Galatsanos,et al.  A similarity learning approach to content-based image retrieval: application to digital mammography , 2004, IEEE Transactions on Medical Imaging.

[2]  David Coppit,et al.  On the Use of Specification-Based Assertions as Test Oracles , 2005, 29th Annual IEEE/NASA Software Engineering Workshop.

[3]  Brad A. Myers,et al.  User interface software tools , 1995, TCHI.

[4]  Capers Jones,et al.  Why software fails , 1996 .

[5]  Steve A. Schneider,et al.  Test Case Preparation Using a Prototype , 1998, B.

[6]  Rob Miller,et al.  GUI testing using computer vision , 2010, CHI.

[7]  Laura M. Yarusso,et al.  Radial gradient-based segmentation of mammographic microcalcifications: observer evaluation and effect on CAD performance. , 2004, Medical physics.

[8]  Atif M. Memon,et al.  Using transient/persistent errors to develop automated test oracles for event-driven software , 2004, Proceedings. 19th International Conference on Automated Software Engineering, 2004..

[9]  José Creissac Campos,et al.  Test case generation from mutated task models , 2011, EICS '11.

[10]  Paul A. Strooper,et al.  Systematically deriving partial oracles for testing concurrent programs , 2001, Proceedings 24th Australian Computer Science Conference. ACSC 2001.

[11]  James Lee Hafner,et al.  Efficient Color Histogram Indexing for Quadratic Form Distance Functions , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Nancy G. Leveson,et al.  An investigation of the Therac-25 accidents , 1993, Computer.

[13]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[14]  Alan F. Smeaton,et al.  Using score distributions for query-time fusion in multimediaretrieval , 2006, MIR '06.

[15]  Juichi Takahashi An automated oracle for verifying GUI objects , 2001, SOEN.

[16]  Atif M. Memon,et al.  Automating regression testing for evolving GUI software , 2005, J. Softw. Maintenance Res. Pract..

[17]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[18]  Atif M. Memon,et al.  Designing and comparing automated test oracles for GUI-based software applications , 2007, TSEM.

[19]  Gabriel L. Zenarosa,et al.  Towards automated oracles for GUI input validation , 2011, AST '11.

[20]  Jason McDonald,et al.  Translating Object-Z specifications to passive test oracles , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).

[21]  Sten Agerholm,et al.  Formal specification and validation at work: a case study using VDM-SL , 1998, FMSP '98.

[22]  Douglas Hoffman A Taxonomy for Test Oracles , 1998 .

[23]  Dianxiang Xu,et al.  A Systematic Capture and Replay Strategy for Testing Complex GUI Based Java Applications , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[24]  Pavel Zezula,et al.  M-tree: An Efficient Access Method for Similarity Search in Metric Spaces , 1997, VLDB.

[25]  S Tangaro,et al.  A completely automated CAD system for mass detection in a large mammographic database. , 2006, Medical physics.

[26]  Mao Ye,et al.  Automated Oracle Based on Multi-Weighted Neural Networks for GUI Testing , 2007 .

[27]  Abdul Rauf,et al.  Ontology driven semantic annotation based GUI testing , 2010, 2010 6th International Conference on Emerging Technologies (ICET).

[28]  M. Young Test Oracles , 2001 .

[29]  Marti A. Hearst,et al.  The state of the art in automating usability evaluation of user interfaces , 2001, CSUR.

[30]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[31]  Rafael C. Gonzales,et al.  Digital Image Processing -3/E. , 2012 .

[32]  Atif M. Memon,et al.  What test oracle should I use for effective GUI testing? , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[33]  Fan Jiang,et al.  GUI Test Case Definition with TTCN-3 , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[34]  B. F. Castro Buschmann, Frank; Meunier, Regine; Rohnert, Hans; Sommerlad, Peter; Stal, Michael. Pattern-oriented software architecture: a system of patterns, John Wiley & Sons Ltd, 1996 , 1997 .

[35]  Mary Lou Soffa,et al.  A comprehensive framework for testing graphical user interfaces , 2001 .

[36]  Franz Wotawa,et al.  GUI savvy end-to-end testing with smart monkeys , 2009, 2009 ICSE Workshop on Automation of Software Test.

[37]  Qing Xie,et al.  Developing cost-effective model-based techniques for GUI testing , 2006, ICSE.

[38]  Yi Wang,et al.  Artificial Neural Network for Automatic Test Oracles Generation , 2008, 2008 International Conference on Computer Science and Software Engineering.

[39]  B. J. Ferro Castro,et al.  Pattern-Oriented Software Architecture: A System of Patterns , 2009 .

[40]  Harry M. Sneed,et al.  Automated Software Quality Assurance , 1985, IEEE Transactions on Software Engineering.

[41]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[43]  J. D. Day,et al.  A test oracle based on formal specifications , 1985 .

[44]  Shih-Fu Chang,et al.  Visually Searching the Web for Content , 1997, IEEE Multim..

[45]  Henry Lieberman,et al.  Aria: an agent for annotating and retrieving images , 2001, Computer.

[46]  Colin Atkinson,et al.  Automated harvesting of test oracles for reliability testing , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[47]  Ana C. R. Paiva,et al.  Towards the Integration of Visual and Formal Models for GUI Testing , 2007, MBT.

[48]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[49]  Jessica Chen,et al.  Specification-based Testing for Gui-based Applications , 2002, Software Quality Journal.

[50]  Richard A. DeMillo,et al.  Completely validated software: test adequacy and program mutation (panel session) , 1989, ICSE '89.

[51]  Brian Marick,et al.  When Should a Test Be Automated , 1998 .

[52]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[53]  Douglas Hoffman,et al.  Using Oracles in Test Automation , 2001 .

[54]  Atif M. Memon,et al.  Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software , 2005, IEEE Transactions on Software Engineering.

[55]  Debra J. Richardson,et al.  Specification-based test oracles for reactive systems , 1992, International Conference on Software Engineering.

[56]  David Lorge Parnas,et al.  Using Test Oracles Generated from Program Documentation , 1998, IEEE Trans. Software Eng..

[57]  Atif M. Memon,et al.  Employing user profiles to test a new version of a GUI component in its context of use , 2006, Software Quality Journal.

[58]  Relatório Técnico,et al.  Estudos Primários e Secundários apoiando a busca por Evidência em Engenharia de Software , 2006 .

[59]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Nuno Vasconcelos,et al.  On the efficient evaluation of probabilistic similarity functions for image retrieval , 2004, IEEE Transactions on Information Theory.

[61]  Elaine J. Weyuker,et al.  On Testing Non-Testable Programs , 1982, Comput. J..

[62]  Claude Caci,et al.  Testing object-oriented systems , 2000, SOEN.

[63]  Johannes Mayer,et al.  On Random Testing of Image Processing Applications , 2006, 2006 Sixth International Conference on Quality Software (QSIC'06).

[64]  Atif M. Memon,et al.  Empirical evaluation of the fault-detection effectiveness of smoke regression test cases for GUI-based software , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[65]  Maryellen L. Giger,et al.  Computer-aided diagnosis of breast lesions in medical images , 2000, Comput. Sci. Eng..

[66]  Hong Zhu A note on test oracles and semantics of algebraic specifications , 2003, Third International Conference on Quality Software, 2003. Proceedings..

[67]  Atif M. Memon An event-flow model of GUI-based applications for testing: Research Articles , 2007 .

[68]  Douglas C. Schmidt,et al.  Object-oriented application frameworks , 1997, CACM.

[69]  Ahmad-Reza Sadeghi,et al.  A Pattern for Secure Graphical User Interface Systems , 2009, 2009 20th International Workshop on Database and Expert Systems Application.

[70]  Mary Lou Soffa,et al.  Automated test oracles for GUIs , 2000, SIGSOFT '00/FSE-8.

[71]  T. H. Tse,et al.  Piping Classification to Metamorphic Testing: An Empirical Study towards Better Effectiveness for the Identification of Failures in Mesh Simplification Programs , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[72]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[73]  Francis C. M. Lau,et al.  Testing object-oriented industrial software without precise oracles or results , 2007, CACM.

[74]  Harald C. Gall,et al.  Constructing CORBA-supported oracles for testing: a case study in automated software testing , 2002, Proceedings 17th IEEE International Conference on Automated Software Engineering,.

[75]  Michael D. Smith,et al.  Persistent Code Caching: Exploiting Code Reuse Across Executions and Applications , 2007, International Symposium on Code Generation and Optimization (CGO'07).

[76]  David Lorge Parnas,et al.  Generating a test oracle from program documentation: work in progress , 1994, ISSTA '94.

[77]  Paula Gomes Mian,et al.  Systematic Review in Software Engineering , 2005 .

[78]  R L Sperling,et al.  "What a long, strange trip it's been". , 2000, Home healthcare nurse.

[79]  Michael Stonebraker,et al.  Chabot: Retrieval from a Relational Database of Images , 1995, Computer.

[80]  Atif M. Memon,et al.  DART: a framework for regression testing "nightly/daily builds" of GUI applications , 2003, International Conference on Software Maintenance, 2003. ICSM 2003. Proceedings..

[81]  Agma J. M. Traina,et al.  How to add content-based image retrieval capability in a PACS , 2002, Proceedings of 15th IEEE Symposium on Computer-Based Medical Systems (CBMS 2002).

[82]  Márcio Eduardo Delamaro,et al.  Using concepts of content‐based image retrieval to implement graphical testing oracles , 2013, Softw. Test. Verification Reliab..