What a Long, Strange Trip It's Been: Past, Present, and Future Perspectives on Software Testing Research

Over the past 25 years the Brazilian Symposium on Software Engineering (SBES) has evolved to become the most important event on software engineering in Brazil. Throughout these years, SBES has gathered a large body of studies in software testing. Aimed at providing an insightful understanding of what has already been published in such event, we synthesized its rich 25-year history of research on software testing. Using information drawn from this overview we attempted to highlight which types of study have been the most applied for conveying software testing efforts. We also devised a co-authorship network to obtain a bird's-eye view of which research groups and scholars have been the most prolific ones. Moreover, by performing a citation analysis of the selected studies we set out to ascertain the importance of SBES in a wider scenario. Finally, borne out by the information extracted from the studies, we shed some light on the state-of-the-art of software testing in Brazil and provide an outlook on its foreseeable future.

[1]  Francisco Brasileiro,et al.  GridUnit : Using the Computational Grid to Speed up Software Testing , 2005 .

[2]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[3]  Adilya P. Malhur,et al.  Evaluation of The Cost of Alternate Mutation Strategies , 1999 .

[4]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[5]  Auri Marcelo Rizzo Vincenzi,et al.  Static Analysis of Java Bytecode for Domain-specific Software Testing ∗ , 2007 .

[6]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures , 2005 .

[7]  Mary Jean Harrold,et al.  Testing: a roadmap , 2000, ICSE '00.

[8]  Alexandre Petrenko,et al.  Experimental Evaluation of Coverage Criteria for FSM-based Testing , 2007, SBES.

[9]  Eliane Martins,et al.  Plavis / FSM : an Environment to Integrate FSM-based Testing Tools , 2005 .

[10]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[11]  José Carlos Maldonado,et al.  Teste estrutural de software: uma abordagem para aplicações de bando de dados relacional , 2000 .

[12]  Francisco Brasileiro,et al.  Obtaining Trustworthy Test Results in Multi-threaded Systems , 2008, SBES.

[13]  E. Martins,et al.  Automatic test data generation for path testing using a new stochastic algorithm , 2005, Anais do XIX Simpósio Brasileiro de Engenharia de Software (SBES 2005).

[14]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[15]  Tatiana Sugeta.  PROTEUM - RS/ST: uma ferramenta para apoiar a validação de especificações statecharts baseada na análise de mutantes. , 1999 .

[16]  Roel Wieringa,et al.  Requirements engineering paper classification and evaluation criteria: a proposal and a discussion , 2005, Requirements Engineering.

[17]  Mario Jino,et al.  Proteum: uma ferramenta de teste baseada na analise de mutantes , 1993 .

[18]  Silvia Regina Vergilio,et al.  Selection and Evaluation of Test Data Based on Genetic Programming , 2003, Software Quality Journal.

[19]  José Carlos Maldonado,et al.  Geração de Dados de Teste , 2007 .

[20]  Rodrigo Dias Rivera.  Proteste : ambiente de apoio ao teste de programas , 1993 .

[21]  Otávio Augusto Lazzarini Lemos,et al.  Pairwise structural testing of object and aspect-oriented Java programs ∗ , 2007 .

[22]  Paulo Borba,et al.  Automated Test Execution Effort Estimation Based on Functional Test Specifications , 2008 .

[23]  André Dantas Rocha.  Uma ferramenta baseada em aspectos para apoio ao teste funcional de programas Java , 2005 .

[24]  Lionel C. Briand,et al.  Empirical studies of software testing techniques: challenges, practical strategies, and future research , 2004, SOEN.

[25]  Márcio Eduardo Delamaro,et al.  Built-In Structural Testing of Web Services , 2010, 2010 Brazilian Symposium on Software Engineering.

[26]  Diogo N. Campanha,et al.  Teste de mutação nos paradigmas procedimental e oo: uma avaliação no contexto de estrutura de dados , 2010 .

[27]  Paulo Cesar Masiero,et al.  Integration testing of aspect-oriented programs : a characterization study to evaluate how to minimize the number of stubs , 2007 .

[28]  Adenilso da Silva Simão,et al.  MuDeL: a language and a system for describing and generating mutants , 2001, J. Braz. Comput. Soc..

[29]  Aditya P. Mathur,et al.  Foundations of Software Testing , 2007 .

[30]  Eliane Martins,et al.  Generalized Extremal Optimization: a competitive algorithm for test data generation , 2007, SBES.

[31]  Mario Jino,et al.  Data flow based integration testing , 1997 .

[32]  José Carlos Maldonado,et al.  PROTEUM/IM: uma ferramenta de apoio ao teste de integração , 1997 .

[33]  Adenilso da Silva Simão,et al.  A Family of Coverage Testing Criteria for Coloured Petri Nets , 2003, Anais do XVII Simpósio Brasileiro de Engenharia de Software (SBES 2003).

[34]  Luciano Bathaglini Biasi.  Geração automatizada de drivers e stubs de teste para JUnit a partir de especificações U2TP , 2006 .

[35]  F. Ferreira,et al.  TaRGeT : a Model Based Product Line Testing Tool , 2010 .

[36]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures [The volume is the outcome of a research seminar that was held in Schloss Dagstuhl in January 2004] , 2005, Model-Based Testing of Reactive Systems.

[37]  E. Barbosa.  Uma contribuição para determinação de um conjunto essencial de operadores de mutação no teste de programas C , 1998 .

[38]  Joseph Robert Horgan,et al.  /spl chi/SUDS-SDL: a tool for diagnosis and understanding software specifications , 1999, Proceedings Sixth Asia Pacific Software Engineering Conference (ASPEC'99) (Cat. No.PR00509).

[39]  Alexandre Ceolin Hausen,et al.  ValiMPI: Uma Ferramenta de Teste Estrutural para Programas Paralelos em Ambiente de Passagem de Mensagem , 2005 .

[40]  M. Pezzè,et al.  Mutant Operators for Testing Concurrent Java Programs , 2001, Anais do XV Simpósio Brasileiro de Engenharia de Software (SBES 2001).

[41]  Mario Jino,et al.  Geração automatica de dados e tratamento de não executabilidade no teste estrutural de software , 1999 .

[42]  Natalia Juristo Juzgado,et al.  Maturing Software Engineering Knowledge through Classifications: A Case Study on Unit Testing Techniques , 2009, IEEE Transactions on Software Engineering.

[43]  Eliane Martins,et al.  FireWeb : uma ferramenta de suporte aos testes de regressão de aplicações Web , 2003 .

[44]  Phil McMinn,et al.  Search-based software test data generation: a survey: Research Articles , 2004 .