Software Testing: Testing New Software Paradigms and New Artifacts

Software testing research is continuously moving toward different challenges and obstacles driven by different forces. What we believe are two main forces that drive software testing evolution are the progress in software development paradigms and languages, and the artefacts along the software development process to be used for testing purposes. Based on this specific perspective, this work provides an ample presentation of issues and assessments related to testing software. Ongoing and (expected) future research directions on software testing are also outlined. Keywords: software testing; code-based testing; object-oriented testing; component-based testing; software architecture-based testing; specification-based testing; formal testing; model-based testing; MDA testing; testing software product lines; testing service-oriented systems

[1]  Mary Jean Harrold,et al.  Testing: a roadmap , 2000, ICSE '00.

[2]  Axel van Lamsweerde,et al.  Formal specification: a roadmap , 2000, ICSE '00.

[3]  Genny Tortora,et al.  Emerging Methods, Technologies and Process Management in Software Engineering , 2008 .

[4]  Jan Tretmans,et al.  Test Generation with Inputs, Outputs and Repetitive Quiescence , 1996, Softw. Concepts Tools.

[5]  Henry Muccini,et al.  Using software architecture for code testing , 2004, IEEE Transactions on Software Engineering.

[6]  A. Jefferson Offutt,et al.  Deriving tests from software architectures , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[7]  Jeffrey O. Kephart,et al.  The Vision of Autonomic Computing , 2003, Computer.

[8]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[9]  Patrick Donohoe.  Introduction to Software Product Lines , 2008, SPLC.

[10]  Clemens A. Szyperski,et al.  Component software - beyond object-oriented programming , 2002 .

[11]  John D. McGregor,et al.  A Practical Guide to Testing Object-Oriented Software , 2001, Addison Wesley object technology series.

[12]  Lori L. Pollock,et al.  The Construction of Contextual Def-Use Associations for Object-Oriented Systems , 2003, IEEE Trans. Software Eng..

[13]  Bev Littlewood,et al.  Modeling the Effects of Combining Diverse Software Fault Detection Techniques , 2000, IEEE Trans. Software Eng..

[14]  Nicolas Belloir,et al.  Component behavior prediction and monitoring through built-in test , 2003, 10th IEEE International Conference and Workshop on the Engineering of Computer-Based Systems, 2003. Proceedings..

[15]  Ina Schieferdecker,et al.  Model-Driven Testing: Using the UML Testing Profile , 2007 .

[16]  Linda M. Northrop,et al.  An Introduction to Software Product Lines , 2004, SPLC.

[17]  Andrea Polini,et al.  Testing software components for integration: a survey of issues and techniques , 2007, Softw. Test. Verification Reliab..

[18]  Volker Gruhn,et al.  State of the art in testing components , 2003, Third International Conference on Quality Software, 2003. Proceedings..

[19]  Alexander L. Wolf,et al.  Software testing at the architectural level , 1996, ISAW '96.

[20]  E. F. Miller,et al.  Ct coverage — initial results , 1993, Software Quality Journal.

[21]  Dolores R. Wallace,et al.  Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric , 1996 .

[22]  Antonia Bertolino,et al.  Software Testing Research and Practice , 2003, Abstract State Machines.

[23]  Henry Muccini,et al.  Towards Testing Product Line Architectures , 2003, TACoS.

[24]  Ina Schieferdecker,et al.  The UML 2.0 Testing Profile and Its Relation to TTCN-3 , 2003, TestCom.

[25]  Jüri Vain,et al.  Model-Based Testing of Reactive Systems , 2009 .

[26]  Luciano Baresi,et al.  Test and Analysis of Web Services , 2007, Test and Analysis of Web Services.

[27]  Edsger W. Dijkstra,et al.  Chapter I: Notes on structured programming , 1972 .

[28]  Colin Atkinson,et al.  Built-in Contract Testing in Model-driven , Component-Based Development , 2002 .

[29]  Ye Wu,et al.  Testing and Quality Assurance for Component-Based Software , 2003 .

[30]  Clemens Szyperski,et al.  Component software , 1997 .

[31]  Richard H. Carver,et al.  Use of sequencing constraints for specification-based testing of concurrent programs , 1998 .

[32]  W. Eric Wong,et al.  Component-Based Software: An Overview of Testing , 2003, Component-Based Software Quality.

[33]  Brian Marick,et al.  EXPERIENCE WITH THE COST OF DIFFERENT COVERAGE GOALS FOR TESTING , 1991 .

[34]  Pankaj Mudholkar,et al.  Software Testing , 2002, Computer.

[35]  R. G. de Vries.  Towards Formal Test Purposes , 2001 .

[36]  Ron Patton,et al.  Software Testing , 2000 .

[37]  Richard N. Taylor,et al.  Structural Testing of Concurrent Programs , 1992, IEEE Trans. Software Eng..

[38]  Stephen H. Edwards,et al.  A framework for practical, automated black‐box testing of component‐based software † , 2001, Softw. Test. Verification Reliab..

[39]  Gilles Bernot,et al.  Testing Against Formal Specifications: A Theoretical View , 1991, TAPSOFT, Vol.2.

[40]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[41]  Juha Taina,et al.  Product family testing: a survey , 2004, SOEN.

[42]  Paul Clements,et al.  Software product lines - practices and patterns , 2001, SEI series in software engineering.

[43]  M. Momotko,et al.  Component+ built-in testing a technology for testing software components , 2004 .

[44]  J Hayhurst Kelly,et al.  A Practical Tutorial on Modified Condition/Decision Coverage , 2001 .

[45]  Elaine J. Weyuker,et al.  An Applicable Family of Data Flow Testing Criteria , 1988, IEEE Trans. Software Eng..

[46]  Christian Schallhart,et al.  Tools for Test Case Generation , 2004, Model-Based Testing of Reactive Systems.

[47]  Debra J. Richardson,et al.  Scenario-based and State Machine-based Testing : An Evaluation of Automated Approaches , 2006 .

[48]  Hernán C. Melgratti,et al.  Testing Service Composition , 2007 .

[49]  Bogdan Korel.  Black-box understanding of COTS components , 1999, Proceedings Seventh International Workshop on Program Comprehension.

[50]  Yingxu Wang,et al.  A method for built-in tests in component-based software maintenance , 1999, Proceedings of the Third European Conference on Software Maintenance and Reengineering (Cat. No. PR00090).

[51]  Alexander L. Wolf,et al.  Acm Sigsoft Software Engineering Notes Vol 17 No 4 Foundations for the Study of Software Architecture , 2022 .

[52]  David S. Rosenblum.  Challenges in Exploiting Architectural Models for Software Testing , 1998 .

[53]  Henry Muccini,et al.  Monitoring Architectural Properties in Dynamic Component-Based Systems , 2007, CBSE.

[54]  Mauro Pezzè,et al.  Testing object-oriented software , 2004, Proceedings. 26th International Conference on Software Engineering.

[55]  Jan Bosch,et al.  Representing Variability in Software Product Lines: A Case Study , 2002, SPLC.

[56]  Henry Muccini,et al.  Software architecture-based regression testing , 2006, J. Syst. Softw..

[57]  Bruno Legeard,et al.  A taxonomy of model-based testing , 2006 .

[58]  Volker Gruhn,et al.  Testing Commercial-off-the-shelf Components And Systems , 2005 .

[59]  A Straw,et al.  Guide to the Software Engineering Body of Knowledge , 1998 .

[60]  Robert M. Poston.  Automating specification-based software testing , 1996 .

[61]  Roland Groz,et al.  Eight Years of Experience in Test Generation from FDTs using TVEDA , 1997, FORTE.

[62]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[63]  Edward V. Berard,et al.  Testing Object-Oriented Software , 2021, TOOLS.

[64]  Janusz W. Laski,et al.  A Data Flow Oriented Program Testing Strategy , 1983, IEEE Transactions on Software Engineering.

[65]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[66]  John D. McGregor,et al.  Testing a Software Product Line , 2001, PSSE.

[67]  Alexandre Petrenko,et al.  Protocol testing: review of methods and relevance for software testing , 1994, ISSTA '94.

[68]  Michael R. Donat.  Automating Formal Specification-Based Testing , 1997, TAPSOFT.

[69]  Alessandro Orso,et al.  Scaling regression testing to large software systems , 2004, SIGSOFT '04/FSE-12.

[70]  Alessandro Orso,et al.  Regression test selection for Java software , 2001, OOPSLA '01.

[71]  Eliane Martins,et al.  Constructing self-testable software components , 2001, 2001 International Conference on Dependable Systems and Networks.

[72]  Jens Grabowski,et al.  From Design to Test with UML: Applied to a Roaming Algorithm for Bluetooth Devices , 2004, TestCom.

[73]  Jan Tretmans,et al.  Testing Concurrent Systems: A Formal Approach , 1999, CONCUR.

[74]  Thierry Jéron,et al.  A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2005 .

[75]  Thierry Jéron,et al.  TGV : theory , principles and algorithms A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2004 .

[76]  Sigrid Eldh.  Software Testing Techniques , 2007 .

[77]  Ye Wu,et al.  Techniques for testing component-based software , 2001, Proceedings Seventh IEEE International Conference on Engineering of Complex Computer Systems.

[78]  Thierry Jéron,et al.  An Experiment in Automatic Generation of Test Suites for Protocols with Verification Technology , 1997, Sci. Comput. Program..

[79]  Mary Jean Harrold.  Architecture-Based Regression Testing of Evolving Systems , 1998 .

[80]  Gregg Rothermel,et al.  Performing data flow testing on classes , 1994, SIGSOFT '94.

[81]  Mauro Pezzè,et al.  Software testing and analysis - process, principles and techniques , 2007 .

[82]  Alessandro Orso,et al.  Automated Testing of Classes , 2000, ISSTA '00.

[83]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[84]  Andrea Polini,et al.  A User-Oriented Framework for Component Deployment Testing , 2005 .

[85]  Leonardo Mariani,et al.  Behavior capture and test: automated analysis of component integration , 2005, 10th IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'05).

[86]  Ina Schieferdecker,et al.  From U2TP Models to Executable Tests with TTCN-3 - An Approach to Model Driven Testing , 2005, TestCom.

[87]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures [The volume is the outcome of a research seminar that was held in Schloss Dagstuhl in January 2004] , 2005, Model-Based Testing of Reactive Systems.