Model-based software component testing

Software component testing (SCT) is a proven software engineering approach to evaluating, improving and demonstrating component reliability and quality for producing trusted software components, which is critical to support the success of component-based software engineering. Model-based testing (MBT) of software components enables the utilisation of a consistent model-based approach and specification (e.g. UML models) for effective component development and testing. However, advancing from model-based development to MBT poses certain crucial challenging problems that remain unresolved and hamper the utilisation of SCT/MBT, and further research is thus required to address those problems to achieve the goal of desirable SCT/MBT effectiveness. This thesis has comprehensively reviewed the important concepts, principles, characteristics and techniques of SCT/MBT in the literature to provide a solid foundation for this research and introduced a set of useful new concepts and definitions to form the first major part of the thesis’s original contributions as follows: 1. In the research areas of software components and software component testing: (a) A new comprehensive taxonomy of software component characteristics (b) A new software component definition (c) A new definition of software component testing (d) A useful taxonomy of software component testing techniques (e) A practical taxonomy of component testability improvement approaches 2. In the research areas of model-based testing and UML-based testing: (a) A study of model-based tests (b) A new definition of model-based testing (c) A new test model definition (d) A new definition of UML-based testing (e) A core UML subset for SCT (f) A study and review of use case driven testing and scenario-based testing The principal original contribution of this thesis is to introduce a novel hybrid SCT methodology, called Model-Based Software Component Testing (MBSCT), which consists of five major methodological components, a three-phase testing framework, six main methodological features and six core testing capabilities. In more detail: (1) The Model-Based Integrated SCT process incorporates software component

[1]  Lionel C. Briand,et al.  Assessing, Comparing, and Combining Statechart- based testing and Structural testing: An Experiment , 2007, ESEM 2007.

[2]  Ivar Jacobson,et al.  The unified modeling language reference manual , 2010 .

[3]  L. C. Briand.  A Critical Analysis of Empirical Research in Software Testing , 2007, ESEM 2007.

[4]  Bertrand Meyer,et al.  The grand challenge of trusted components , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[5]  Hong Sun,et al.  Investigating the use of analysis contracts to improve the testability of object‐oriented code , 2003, Softw. Pract. Exp..

[6]  Alexander Pretschner,et al.  Model-Based Testing in Practice , 2005, FM.

[7]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[8]  Edward F. Moore,et al.  Gedanken-Experiments on Sequential Machines , 1956 .

[9]  M. Sarma,et al.  Automatic generation of test specifications for coverage of system state transitions , 2009, Inf. Softw. Technol..

[10]  John Morris,et al.  A software component verification tool , 2000, Proceedings International Conference on Software Methods and Tools. SMT 2000.

[11]  Ivar Jacobson,et al.  The Unified Modeling Language User Guide , 1998, J. Database Manag..

[12]  R. Mall,et al.  UML Sequence Diagram Based Testing Using Slicing , 2005, 2005 Annual IEEE India Conference - Indicon.

[13]  Rance Cleaveland,et al.  Using formal specifications to support testing , 2009, CSUR.

[14]  J. Voas,et al.  Software Testability: The New Verification , 1995, IEEE Softw..

[15]  George T. Heineman,et al.  Component-Based Software Engineering: Putting the Pieces Together , 2001 .

[16]  Jerry Zeyu Gao,et al.  A component testability model for verification and measurement , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[17]  Jean Hartmann,et al.  A UML-based approach to system testing , 2005, Innovations in Systems and Software Engineering.

[18]  Mark Blackburn,et al.  Why Model-Based Test Automation is Different and What You Should Know to Get Started , 2004 .

[19]  G. Bundell,et al.  Software Component Certification , 2001, Computer.

[20]  Guilherme Horta Travassos,et al.  Porantim: An approach to support the combination and selection of Model-based Testing techniques , 2009, 2009 ICSE Workshop on Automation of Software Test.

[21]  G. King,et al.  On testable object-oriented programming , 1997, SOEN.

[22]  James A. Whittaker,et al.  Model‐Based Software Testing , 2002 .

[23]  Massimiliano Di Penta,et al.  Assessing and improving state-based class testing: a series of experiments , 2004, IEEE Transactions on Software Engineering.

[24]  John Daniels,et al.  UML Components: A Simple Process for Specifying Component-Based Software , 2000 .

[25]  Ina Schieferdecker,et al.  The UML 2.0 Testing Profile and Its Relation to TTCN-3 , 2003, TestCom.

[26]  Falk Fraikin,et al.  SeDiTeC-testing based on sequence diagrams , 2002, Proceedings 17th IEEE International Conference on Automated Software Engineering,.

[27]  C. M. Sperberg-McQueen,et al.  Extensible Markup Language (XML) , 1997, World Wide Web J..

[28]  Eda Marchetti,et al.  Integration of "Components" to Test Software Components , 2003, TACoS.

[29]  Mauro Pezzè,et al.  Software testing and analysis - process, principles and techniques , 2007 .

[30]  John Morris,et al.  Setting a Framework for Trusted Component Trading , 2003, Component-Based Software Quality.

[31]  Anneke Kleppe,et al.  The Object Constraint Language: Getting Your Models Ready for MDA , 2003 .

[32]  Antonio Vallecillo,et al.  Component-based software quality : methods and techniques , 2003 .

[33]  Jan Tretmans,et al.  Model-Based Testing and Some Steps towards Test-Based Modelling , 2011, SFM.

[34]  Thomas Erl,et al.  Service-Oriented Architecture: Concepts, Technology, and Design , 2005 .

[35]  Axel Uhl,et al.  Model-Driven Architecture , 2002, OOIS Workshops.

[36]  Markus Völter.  A Taxonomy of Components , 2003, J. Object Technol..

[37]  Stuart Kent,et al.  Model Driven Engineering , 2002, IFM.

[38]  Antonia Bertolino,et al.  Software Testing Research and Practice , 2003, Abstract State Machines.

[39]  Ferhat Khendek,et al.  Test Selection Based on Finite State Models , 1991, IEEE Trans. Software Eng..

[40]  Sudipto Ghosh,et al.  Test Input Generation Using UML Sequence and State Machines Models , 2009, 2009 International Conference on Software Testing Verification and Validation.

[41]  Lionel C. Briand,et al.  Automated support for deriving test requirements from UML statecharts , 2005, Software & Systems Modeling.

[42]  R. Mall,et al.  Automatic Test Case Generation from UML Sequence Diagram , 2007, 15th International Conference on Advanced Computing and Communications (ADCOM 2007).

[43]  Grady Booch,et al.  Book Reviews , 1936, Computer.

[44]  Mark Harman,et al.  Formal Methods and Testing, An Outcome of the FORTEST Network, Revised Selected Papers , 2008, Formal Methods and Testing.

[45]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[46]  M. Scheetz,et al.  Generating test cases from an OO model with an AI planning system , 1999 .

[47]  Sudipto Ghosh,et al.  Using UML Sequence Diagrams and State Machines for Test Input Generation , 2008, 2008 19th International Symposium on Software Reliability Engineering (ISSRE).

[48]  Jan Tretmans,et al.  Model Based Testing with Labelled Transition Systems , 2008, Formal Methods and Testing.

[49]  Lionel C. Briand,et al.  Empirical studies of software testing techniques: challenges, practical strategies, and future research , 2004, SOEN.

[50]  Roy S. Freedman,et al.  Testability of Software Components , 1991, IEEE Trans. Software Eng..

[51]  Alexander Pretschner,et al.  Methodological Issues in Model-Based Testing , 2004, Model-Based Testing of Reactive Systems.

[52]  Doo-Hwan Bae,et al.  Test cases generation from UML state diagrams , 1999, IEE Proc. Softw..

[53]  Robert B. France,et al.  Test adequacy criteria for UML design models , 2003, Softw. Test. Verification Reliab..

[54]  B. Kitchenham,et al.  Case Studies for Method and Tool Evaluation , 1995, IEEE Softw..

[55]  Robert M. Hierons,et al.  Testing from a Z Specification , 1997, Softw. Test. Verification Reliab..

[56]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[57]  Andrea Polini,et al.  WCT: A Wrapper for Component Testing , 2002, FIDJI.

[58]  Eda Marchetti,et al.  The Cow_Suite Approach to Planning and Deriving Test Suites in UML Projects , 2002, UML.

[59]  Raymond A. Paul,et al.  Scenario-based functional regression testing , 2001, 25th Annual International Computer Software and Applications Conference. COMPSAC 2001.

[60]  Jan Tretmans,et al.  Conformance Testing with Labelled Transition Systems: Implementation Relations and Test Generation , 1996, Comput. Networks ISDN Syst..

[61]  Jens Grabowski,et al.  From Design to Test with UML: Applied to a Roaming Algorithm for Bluetooth Devices , 2004, TestCom.

[62]  John Morris,et al.  Using symbolic execution to guide test generation , 2005, Softw. Test. Verification Reliab..

[63]  M.W. Raza.  Comparison of class test integration ordering strategies , 2005, Proceedings of the IEEE Symposium on Emerging Technologies, 2005..

[64]  Jonas Boberg.  Early fault detection with model-based testing , 2008, ERLANG '08.

[65]  William E. Lorensen,et al.  Object-Oriented Modeling and Design , 1991, TOOLS.

[66]  A. Jefferson Offutt,et al.  UML-Based Integration Testing for Component-Based Software , 2003, ICCBSS.

[67]  Marc J. Balcer,et al.  The category-partition method for specifying and generating fuctional tests , 1988, CACM.

[68]  Clémentine Nebut,et al.  Automatic test generation: a use case driven approach , 2006, IEEE Transactions on Software Engineering.

[69]  David S. Rosenblum.  A Practical Approach to Programming With Assertions , 1995, IEEE Trans. Software Eng..

[70]  Aamer Nadeem,et al.  A state-based approach to integration testing based on UML models , 2007, Inf. Softw. Technol..

[71]  Per Runeson,et al.  Checklists for Software Engineering Case Study Research , 2007, ESEM 2007.

[72]  Gary A. Bundell,et al.  A UML-Based Methodology for Software Component Testing , 2007, IMECS.

[73]  Ye Wu,et al.  Testing and Quality Assurance for Component-Based Software , 2003 .

[74]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[75]  Jeffrey M. Voas.  Quality Time: How Assertions Can Increase Test Effectiveness , 1997, IEEE Softw..

[76]  Beatriz Pérez Lamancha,et al.  Automated model-based testing using the UML testing profile and QVT , 2009, MoDeVVa '09.

[77]  Alexander Pretschner,et al.  One evaluation of model-based testing and its automation , 2005, ICSE.

[78]  Kendall Scott,et al.  UML distilled - a brief guide to the Standard Object Modeling Language (2. ed.) , 2000, notThenot Addison-Wesley object technology series.

[79]  Hans-Gerhard Groß,et al.  Component-based software testing with UML , 2004 .

[80]  Alan Hartman,et al.  Choosing a Test Modeling Language: A Survey , 2006, Haifa Verification Conference.

[81]  Manfred Broy,et al.  What characterizes a (software) component? , 1998, Softw. Concepts Tools.

[82]  Jerry Zeyu Gao,et al.  On Building Testable Software Components , 2002, ICCBSS.

[83]  T.C. Lethbridge,et al.  Guide to the Software Engineering Body of Knowledge (SWEBOK) and the Software Engineering Education Knowledge (SEEK) - a preliminary mapping , 2001, 10th International Workshop on Software Technology and Engineering Practice.

[84]  Wei-Tek Tsai,et al.  Scenario-based object-oriented test frameworks for testing distributed systems , 2003, The Ninth IEEE Workshop on Future Trends of Distributed Computing Systems, 2003. FTDCS 2003. Proceedings..

[85]  Bertrand Meyer,et al.  Object-Oriented Software Construction, 2nd Edition , 1997 .

[86]  A. Jefferson Offutt,et al.  A Comparative Evaluation of Tests Generated from Different UML Diagrams , 2008, 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing.

[87]  Stephen H. Edwards,et al.  Contract-checking wrappers for C++ classes , 2004, IEEE Transactions on Software Engineering.

[88]  Christine Mingins,et al.  Providing Trusted Components to the Industry , 1998, Computer.

[89]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[90]  Clemens A. Szyperski,et al.  Component software - beyond object-oriented programming , 2002 .

[91]  James M. Bieman,et al.  Improving software testability with assertion insertion , 1994, Proceedings., International Test Conference.

[92]  Yingxu Wang,et al.  On built-in test reuse in object-oriented framework design , 2000, CSUR.

[93]  Andrea Polini,et al.  A framework for component deployment testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[94]  Jean Hartmann,et al.  UML-Based integration testing , 2000, ISSTA '00.

[95]  Ivar Jacobson,et al.  The Unified Software Development Process , 1999 .

[96]  Robert B. France,et al.  A Systematic Approach to Generate Inputs to Test UML Design Models , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[97]  Jeffrey M. Voas.  Composing Software Component "ilities" , 2001, IEEE Softw..

[98]  A. Jefferson Offutt,et al.  Generating Tests from UML Specifications , 1999, UML.

[99]  Lionel C. Briand,et al.  Towards Automated Support for Deriving Test Data from UML Statecharts , 2003, UML.

[100]  Alexander Pretschner,et al.  Abstractions for Model-Based Testing , 2005, Electron. Notes Theor. Comput. Sci..

[101]  Jan Tretmans,et al.  Model-Based Testing of Environmental Conformance of Components , 2006, FMCO.

[102]  Wei-Tek Tsai,et al.  Rapid Pattern-Oriented Scenario-Based Testing for Embedded Systems , 2005 .

[103]  Ivar Jacobson,et al.  Unified Modeling Language , 2020, Definitions.

[104]  Mary Jean Harrold,et al.  Testing: a roadmap , 2000, ICSE '00.

[105]  Li Qing,et al.  Test Case Automate Generation from UML Sequence Diagram and OCL Expression , 2007 .

[106]  Edsger W. Dijkstra,et al.  Structured programming , 1972, A.P.I.C. Studies in data processing.

[107]  Robert B. France,et al.  Test adequacy assessment for UML design model testing , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[108]  Robert M. Hierons,et al.  Testing a system specified using Statecharts and Z , 2001, Inf. Softw. Technol..

[109]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[110]  Simon Shim,et al.  Monitoring software components and component-based software , 2000, Proceedings 24th Annual International Computer Software and Applications Conference. COMPSAC2000.

[111]  Alan W. Brown,et al.  The Current State , 2016 .

[112]  Guilherme Horta Travassos,et al.  Model-based testing approaches selection for software projects , 2009, Inf. Softw. Technol..

[113]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[114]  Bruno Legeard,et al.  A taxonomy of model-based testing , 2006 .

[115]  A. Jefferson Offutt,et al.  Using UML Collaboration Diagrams for Static Checking and Test Generation , 2000, UML.

[116]  Carl E. Landwehr,et al.  Basic concepts and taxonomy of dependable and secure computing , 2004, IEEE Transactions on Dependable and Secure Computing.

[117]  Forrest Shull,et al.  Improving Evidence about Software Technologies: A Look at Model-Based Testing , 2008, IEEE Software.

[118]  Clémentine Nebut,et al.  Requirements by contracts allow automated system testing , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[119]  Susan Elliott Sim,et al.  Case studies for software engineers , 2005, Proceedings. 26th International Conference on Software Engineering.

[120]  Jean-Marc Jézéquel,et al.  Making Components Contract Aware , 1999, Computer.

[121]  Wei-Tek Tsai,et al.  Scenario-Based Web Services Testing with Distributed Agents , 2003 .

[122]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[123]  Gary A. Bundell,et al.  Model-Based Software Component Testing: A UML-Based Approach , 2007, 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007).

[124]  Ivica Crnkovic,et al.  Building Reliable Component-Based Software Systems , 2002 .

[125]  Franck Barbier.  COTS Component Testing through Built-In Test , 2005 .

[126]  Shaoying Liu,et al.  Criteria for generating specification-based tests , 1999, Proceedings Fifth IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'99) (Cat. No.PR00434).

[127]  Weiqun Zheng,et al.  Test by Contract for UML-Based Software Component Testing , 2008, International Symposium on Computer Science and its Applications.

[128]  Jeffrey Parsons,et al.  How UML is used , 2006, CACM.

[129]  John Morris,et al.  Determining Component Reliability Using a Testing Index , 2002, ACSC.

[130]  Zhen Ru Dai.  UML 2.0 Testing Profile , 2004, Model-Based Testing of Reactive Systems.

[131]  Jim Davies,et al.  A method for the automatic generation of test suites from object models , 2004, Inf. Softw. Technol..

[132]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[133]  R. Mall,et al.  Automatic test case generation using unified modeling language (UML) state diagrams , 2008, IET Softw..

[134]  Robert V. Binder,et al.  Design for testability in object-oriented systems , 1994, CACM.

[135]  Paul A. Strooper,et al.  Automated Generation of Test Cases Using Model-Driven Architecture , 2007, Second International Workshop on Automation of Software Test (AST '07).

[136]  Wanchai Rivepiboon,et al.  Automated-generating test case using UML statechart diagrams , 2003 .

[137]  Lionel C. Briand,et al.  A UML-Based Approach to System Testing , 2001, Software and Systems Modeling.

[138]  Alfredo Benso,et al.  System-level functional testing from UML specifications in end-of-production industrial environments , 2005, International Journal on Software Tools for Technology Transfer.

[139]  Ina Schieferdecker,et al.  Model-Driven Testing: Using the UML Testing Profile , 2007 .

[140]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[141]  Robert B. France,et al.  Testing UML designs , 2007, Inf. Softw. Technol..

[142]  Robert B. France,et al.  Rigorous Testing by Merging Structural and Behavioral UML Representations , 2003, UML.

[143]  Ina Schieferdecker,et al.  From U2TP Models to Executable Tests with TTCN-3 - An Approach to Model Driven Testing , 2005, TestCom.

[144]  Manfred Broy,et al.  Model-Based Testing of Reactive Systems, Advanced Lectures [The volume is the outcome of a research seminar that was held in Schloss Dagstuhl in January 2004] , 2005, Model-Based Testing of Reactive Systems.

[145]  Raymond A. Paul,et al.  Adaptive scenario-based object-oriented test frameworks for testing embedded systems , 2002, Proceedings 26th Annual International Computer Software and Applications.

[146]  Douglas C. Schmidt,et al.  Guest Editor's Introduction: Model-Driven Engineering , 2006, Computer.

[147]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[148]  Alan Bundy,et al.  Constructing Induction Rules for Deductive Synthesis Proofs , 2006, CLASE.

[149]  Brian Randell,et al.  Fundamental Concepts of Dependability , 2000 .

[150]  簡聰富,et al.  物件導向軟體之架構(Object-Oriented Software Construction)探討 , 1989 .

[151]  Rajesh Subramanyan,et al.  A survey on model-based testing approaches: a systematic review , 2007, WEASELTech '07.

[152]  Dehla Sokenou.  Generating Test Sequences from UML Sequence Diagrams and State Diagrams , 2006, GI Jahrestagung.

[153]  Yingxu Wang,et al.  A method for built-in tests in component-based software maintenance , 1999, Proceedings of the Third European Conference on Software Maintenance and Reengineering (Cat. No. PR00090).

[154]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[155]  Guilherme Horta Travassos,et al.  Surveying model based testing approaches characterization attributes , 2008, ESEM '08.

[156]  A. Pretschner,et al.  Model-based testing for real , 2004, International Journal on Software Tools for Technology Transfer.

[157]  Jean-Marc Jézéquel,et al.  A UML-integrated Test Description Language for Component Testing , 2001, pUML.

[158]  Marc-Florian Wendland,et al.  Using the UML Testing Profile for Enterprise Service Choreographies , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[159]  Kung-Kiu Lau,et al.  Software Component Models , 2007, IEEE Trans. Software Eng..

[160]  Guilherme Horta Travassos,et al.  Supporting the selection of model-based testing approaches for software projects , 2008, AST '08.

[161]  Siddhartha R. Dalal,et al.  Model-based testing in practice , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[162]  David S. Janzen,et al.  Test-driven development concepts, taxonomy, and future direction , 2005, Computer.

[163]  Philip Samuel,et al.  Test Sequence Generation from UML Sequence Diagrams , 2008, 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing.